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ABSTRACT 

Results of an initial attempt at measuring teamwork 
processes using computer simulation are presented, focusing on 
assessing team processes that emerge during the negotiation of a 
contract. The interaction between team members and how that 
interaction affects team performance were explored using a computer 
simulated negotiation. Participants interacted through computers 
using predetermined messages that were categorized as belonging to 
five teamwork processes. Tracking the messages gives a picture of 
teamwork processes. Two pilot studies involving 15 adults established 
the utility of the approach. Hypotheses that team processes would be 
positively associated with team performance and time and negatively 
associated with time-to-agreement and degree of fixed-pie bias were 
tested with 81 high school students. Results indicate that the 
measurement can be accomplished in a reasonably reliable and much 
more time-efficient manner through the computer simulation than 
through previous approaches. Five appendixes provide supplemental 
information. Twenty tables and four figures present study findings. 
(SLD) 
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MEASUREMENT OF TEAMWORK PROCESSES 
USING COMPUTER SIMULATION 1 

Harold F. O'Neil, Jr., USC/CRESST 
Gregory K. Chung, UCLA/CRESST 
Richard S. Brown, UCLA/CRESST 



Introduction 

The National Center for Research on Evaluation, Standards, and Student 
Testing (CRESST) has a collaborative agreement from the Office of Educational 
Research and Improvement to study methodologies for the assessment of 
workforce competencies. In previous reports (O'Neil, Allred, & Dennis, 1992, 
1994) we documented several validation studies of our measures of the 
negotiation subskill of an interpersonal competency (U.S. Department of 
Labor, 1991, 1992). In this report we present results of our initial attempt at 
measuring teamwork processes using computer simulation. Our team, 
outcome measures are the same as our previous research (O'Neil et al., 1994). 

In this section we cover four issues: (a) the importance of teams, (b) a 
taxonomy of teamwork processes, (c) teamwork processes within a negotiation 
context, and (d) measuring teamwork processes. The focus of our effort is on a 
simple but fundamental question: How do we assess team processes and 
outcomes such that the measurement is not only reliable and valid, but also 
timely? 



1 We wish to thank Dr. Eva Baker, Mr. Robert Dennis, Mr. Howard Herl. and Dr. Joan 
Herman of CRESST/ UCLA and Mr. Keith Allred of UCLA for their assistance. We also wish 
to thank the leadership of both the Airconditioning and Refrigeration Contractors Association 
of Southern California, Inc., and the United Association of Journeymen and Apprentices of the 
Plumbing and Pipefitting Industry, Airconditioning and Refrigeration Fitters Division, 
Local No. 250 (AFL-CIO). In particular, wa wish to acknowledge the assistance of union 
member Mr. Jack Ferrara. Finally, we also wish to thank Dr. John Brady, Ms. Susan Hall, 
Ms. Michele Jansen, and their students for their assistance and participation. 
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Importance of Teams 

There has been an increased interest in team performance over the last 20 
years attesting to its importance. Parallel efforts within the educational (e.g., 
Webb & Farivar, 1994; Webb & Palincsar, in press), industrial (e.g., Druckman 
& Bjork, 1994; Sundstrom, Be Meuse, & Futrell, 1990), and military (e.g., 
Franken & O'Neil, 1994; Swezey & Salas, 1992) sectors have attempted to 
characterize, measure, and understand the constructs underlying effective 
teams. Although each sector focuses on different facets of teamwork and 
employs different paradigms, all three sectors recognize the potential of teams 
to increase and enhance learning, task performance, work productivity, and 
product quality. Indeed, one theme that emerges from all three sectors is that 
under optimal conditions, effective teams often reflect the cliche that the whole 
is greater than the sum of its parts. 

Of particular relevance to our work is how changes in American industry- 
are driving changes in education and training. Numerous studies and 
commissions have examined the skill requirements for tomorrow's entry level 
worker. O'Neil, Allred, and Baker (1992) reviewed five major studies on 
workforce readiness commissioned by federal, state, or private agencies. All 
studies identified interpersonal and teamwork skills to be essential skills. 

The private sector's emphasis on teamwork has gained momentum as 
global competitive pressures have forced companies to alter management 
practices. Downsizing, reorganizations, and other structural changes in the 
workplace have resulted in companies placing greater responsibility on their 
workforce. A prevailing belief is that teams offer the potential to dramatically 
improve a company's competitiveness (Cannon-Bowers, Oser, & Flanagan, 
1992). Whether employees participate in teams as adjuncts to their regular job 
(e.g., quality- circle teams) or participate in teams full-time (e.g., negotiating 
teams), the implication is clear: Increasingly, employees are being required to 
be part of teams. Given the importance o^ teams in the American workforce, 
one of our goals is to develop feasible me a . :es of teamwork processes of high 
school students. Educators and employers can then use these measures to 
help them focus on education and training efforts. 
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A Taxonomy of Team Processes 

Morgan, Salas, and Glickman (1993) provide insight into the nature of 
teams. In their model of team development, Morgan et al. (1993) postulate two 
tracks of team process, a taskwork track and a teamwork track. The taskwork 
track accounts for specific activities unique to the task. Taskwork team skills 
influence how well a team performs on a particular task (e.g., whether or not 
a team of negotiators reaches an agreement with the other party). Taskwork 
skills are domain-dependent, task-related activities. The teamwork track or 
team workskills influence how effective an individual member will be as part 
of a team and are domain-independent team skills. Toam workskills 
encompass skills such as adaptability, coordination, cooperation, and 
communication. Effective teams develop competence along both tracks. 
Members of effective teams possess basic skills required for the task and know 
how to coordinate their activities, communicate with each other, and respond 
effectively to changing conditions. 

We have drawn on the work of Morgan et al. (1993), Salas, Dickinson, 
Converse, and Tannenbaum (1992), Burke, Volpe, Cannon-Bowers, and Salas 
(1993), and others (O'Neil, Baker, & Kazlauskas, 1992; Webb, 1993; Webb & 
Palincsar, in press) to aid our development of teamwork process measures. 
From the taxonomy of teamwork skills identified by Burke et al. (1993), we have 
adopted the following six categories: (a) adaptability — recognizing problems 
and responding appropriately, (b) coordination — organizing team activities to 
complete a task on time, (c) decision making — using available information to 
make decisions, (d) interpersonal — interacting cooperatively with other team 
members, (e) leadership — providing direction for the team, and (f) 
communication — the overall exchange of clear and accurate information. In 
the following discussion we first define and then elaborate on each process 
with examples from the teamwork literature. 

Adaptability is the process by which a team is able to monitor the source 
and nature of problems through an awareness of team activities and factors 
bearing on the task. Adaptive teams use this information to adjust to 
situational demands by using compensatory and feedback behaviors. 

An important facet of adaptability is the detection and correction of 
problems. Sometimes this detection process is labeled situational awareness. 
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Members cross-check each other's performance, provide backup when needed, 
and freely provide feedback to and accept feedback from each other (Mclntyre & 
Salas, in press). For example, Oser, McCallum, Salas, and Morgan (1989) 
observed moderate correlations between various team behaviors and team 
effectiveness as measured by a team simulation score. Oser et al. (1989) 
studied the performance of 13 Navy fire-control crews and found that the top 
three (more effective) teams, compared to bottom three (less effective) teams, 
differed across a range of behaviors. With respect to problem detection and 
correction, members of more effective teams (a) helped others who were 
having difficulty with a task (r = .67, p < .05), (b) assisted others when they had 
difficult tasks to perform (r = .49, p < .05), (c) suggested to others to recheck 
their work to find their mistakes (r = .54, p < .05), and (d) provided suggestions 
to others on the best way to find an error (r = .49, p < .05). 

Coordination is the process by which team resources, activities, and 
responses are organized to ensure that tasks are integrated, synchronized, 
and completed within established temporal constraints. Our view of 
coordination is that it primarily involves task accomplishment rather than 
interpersonal harmony. 

One area where coordination has received much attention is in aircrew 
management. Poor team coordination has been cited as a cause in aircraft 
disasters (Foushee, 1984) while good team coordination has been associated 
with better training performance (Brannick, Roach, & Salas, 1993; Kanki, 
Lozito, & Foushee, 1989a). For example, Brannick et al. (1993) had 52 dyads 
participate in a low-fidelity (personal computer) flight simulator. Participants 
were college students participating for extra class credit. Most participants 
(76%) did not have any personal computer flight simulator experience. The 
teams' mission was to shoot down as many enemy planes as possible within 30 
minutes. Each member had control over different parts of the simulator (e.g., 
one member would control the piloting and the other would control the fire- 
control system). Interdependency was built into the task such that the enemy 
plane could be shot down only with both members working together. With 
respect to coordination, Brannick et al. (1993) found significant correlations 
between team coordination and the outcome measure of team performance (as 
measured by the number of radar "locks" by the enemy plane; r = .29, p < .05). 
Brannick et al. defined coordination as properly sequenced behavior and the 
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exchange of useful information. Oser et al. (1989) observed similar results of 
effective information management. Members of effective teams, rompared to 
less effective teams, showed more coordinated gathering of information (r = 
.50, p < .05). 

Interestingly, Brannick et al. (1993) also found that experience v/ith 
computer flight simulators was significantly correlated with coordination (r = 
.31, p < .05), as well as with (a) communicating about escaping the enemy 
plane (r = -.34, p < .05), and (b) communicating about the meaning of the 
simulator display (r = -.58, p < .05). It is interesting that the latter two 
relationships are negative. One interpretation of their finding is that 
familiarity of the situation contributes to how team members coordinate their 
activities. Team members share an understanding about certain tactical and 
operational elements of the simulator, which reduces the need to discuss those 
elements. Such shared mental models have been observed in aircrews 
familiar with each other compared to aircrews that were working together for 
the first time (Foushee, Lauber, Baetge, & Acomb, 1986). Foushee et al. (1986) 
speculated that familiarity aided team performance as crew members were 
familiar with each other's behavioral and communication styles. This 
familiarity helped crew members anticipate and respond to each other's needs 
and actions. Similarly, Kanki, Lozito, and Foushee (1989b) observed that 
predictable and homogeneous communication patterns distinguished low- 
error from high-error aircrews using high-fidelity flight simulators. Low- 
error aircrews exhibited uniform communication patterns with respect to 
speaking and responding to each other. Conversely, high-error aircrews 
exhibited widely varying patterns. In effect, these aircrews lacked a standard 
way of communicating with each other, in contrast to the low-error aircrews. 

The nature of these kinds of shared mental models has been the focus of 
recent models of team performance (Cannon-Bowers, Salas, & Converse, 1993; 
Rouse, Cannon-Bowers, & Salas, 1992) and is discussed in greater detail in th<i 
next section. 

Decision making is the ability to integrate information, use logical and 
sound judgments, identify possible alternatives, select the best solution, and 
evaluate the consequences. 
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Shared mental models provide a useful perspective on team decision 
making (Cannon-Bowers et al., 1993; Rouse et al., 1992). A mental model can 
be thought of as a person's conception of a system (e.g., a negotiation task). 
Mental models help people describe the purpose and form of the system, help 
them explain the system's function and what it is doing, and help them predict 
what the system will do in the future (Rouse et al., 1992). In terms of team 
performance, Cannon-Bowers et al. (1993) suggest that four kinds of mental 
models are pertinent to team performance: (a) models of the task to be 
performed; (b) models of the operating characteristics of the task-related 
equipment; (c) models of the roles, responsibilities, and interdependencies of 
each team member, and of how information flows within the team; and 
(d) models of each member's skills, knowledge, abilities, and behavior 
patterns. A shared mental model represents an overlap between individual 
members' mental models. 

Rouse et al. (1992) make several predictions about teams that possess 
shared mental models. Two are relevant to decision making. First, Rouse et 
al. (1992) predict that teams with shared mental models will require less overt 
planning time for good performance. Essentially, if everyone knows what to 
do. then they don't need to discuss it. Second, teams with shared mental 
models will engage in less overt communication but maintain performance. 
In this case, team members are competent and proficient in what they do and 
how they interface with each other. Overt requests for information will 
decrease because team members are able to predict the needs and actions of 
others. What the shared mental model perspective suggests is that effective 
decision making is inextricably tied to group coordination. Presumably, the 
quality of decision making covaries with the quality of coordination. The more 
coordinated the group, the better the decision-making process and vice-versa. 

These predictions have an important implications for team decision 
making. Teams with shared mental models will converge on a decision faster 
because everyone has a comparable understanding of the task demands. 
Team members capitalize on their shared understanding to minimize 
nonessential, overt communication. Everyone knows who is responsible for 
what and who to communicate with. With a shared understanding members 
know whom to provide information to and when to provide it, and the recipient 
of information knows that the information is relevant and timely. 
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Support for this view is observed in Brannick et al.'s (1993) study- 
discussed earlier. With respect to time, Brannick et al. observed significant 
correlations between task time and quality of performance as measured by (a) 
observers' holistic ratings (r = -.48, p < .05) and (b) number of locks by the 
enemy aircraft (r = .53, p < .05). Teams that took longer to shoot down the 
enemy plane were (a) rated as performing poorer on the simulation and (b) 
had a higher number of radar locks on their plane by the enemy. And as 
discussed earlier, Brannick et al. (1993) also found significant negative 
correlations between experience and two communication frequency measures, 
and found negative correlations (although these correlations were 
nonsignificant) between experience and five of the remaining six indicators. 
Further, Brannick et al. (1993) found negative correlations between all 
communication acts and the outcome measures, suggesting that the more 
team members "talked it up," the more their task performance suffered. The 
research conducted by Lahey and Slough (1982) suggests this interpretation in 
that instructors who graded team training exercises gave lower grades to 
military teams doing excessive talking. 

These negative correlations between communication frequency and team 
performance is puzzling, given the wealth of findings pointing toward the 
importance of communicating on teamwork (Foushee et al., 1986; Kanki et al., 
1989b; Mclntyre & Salas, in press). What may be occurring is that poorly 
performing teams are "talking it up" in an attempt to improve their condition. 

Interpersonal skill is the ability to improve the quality of team member 
interactions through the resolution of team members' dissent, or the use of 
cooperative behaviors. 

Team reinforcing behaviors have been found to be related to effective team 
performance (Oser et al., 1989). Interpersonal processes are important not so 
much to minimize intergroup friction, but more so to foster team 
interdependence. That is, cooperative behavior reflects a commitment to team 
performance (Mullen & Copper, 1994). Cooperative behavior reflects a belief 
that each member is critical to the overall success of the team, and helping 
others helps the team. Members of effective teams value team performance 
over individual performance and view themselves as part of a team and not 
simply individuals working together (Mclntyre & Salas, in press). 
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Oser et al. (1989) found significant relationships between interpersonal 
behaviors and team performance. Members of more effective teams tended to 
(a) make positive, motivating statements (r = .54, p < .05), (b) praise others for a 
good job (r = .59, p < .05), and (c) thank others for catching a mistake (r = .60, p 
< .05). In addition, Oser et al. (1989) observed negative correlations when team 
members made negative statements directed at the team or the training (r = 
-.70), or when a member raised his voice when correcting another member (r = 
-.60, p < .05). Brannick et al. (1993) found a similar pattern of results. In their 
study, Brannick et al. (1993) found significant relationships between (a) the 
amount of suggestions given and team performance (r = .29, p < .05), and (b) 
cooperative behavior and team performance (r = .29, p < .05). 

Leadership is the ability to direct and coordinate the activities of other 
team mtxnbers, assess team performance, assign tasks, plan and organize, 
and establish a positive atmosphere. 

In their summary of "lessons learned" from their teamwork research, 
Mclntyre and Salas (in press) suggest that team leadership style plays a 
crucial role in the functioning of a team. In particular, good team leaders 
exhibit the kinds of behaviors that help team performance in general: the 
ability to ac ipt to changing conditions, exchange information, provide and 
accept feedback, and provide and accept help. A good team leader is a model of 
teamwork that other team members can emulate. 

The Oser et al. (1989) study described earlier provides support to Mclntyre 
and Salas' (in press) analysis that good team leaders reflect good team 
processes. Oser et al. (1989) observed that leaders in more effective teams, 
compared to leaders in less effective teams, initiated far more of the critical 
behaviors (i.e., error identification and resolution, coordinating information 
gathering, and team encouragement). In their review of the teamwork 
literature, Macpherson and Perez (1992) concluded that during training, a 
team leader should be encouraged to solicit recommendations from team 
members, analyze and comment on those recommendations, and then 
communicate his or her decisions to the team. 

One crucial aspect of good team leadership is the willingness to accept 
feedback from subordinates. A good team leader establishes an environment 
where there is a free and bi-directional flow of information between the leader 
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and members. Leaders who constrain this flow of information — by virtue of 
their leadership style, rank, or personality — impede team performance. Team 
members in this kind of environment are often inhibited from giving feedback 
or backup. Foushee (1984) reports that copilots in these kinds of environments 
often hesitate to voice concerns, even in potentially dangerous situations. For 
example, in 1079 a commuter plane crashed because the copilot failed to 
assume command after the pilot became incapacitated. The captain was the 
company vice-president, had a gruff personality, and was noticeably upset on 
the day of the crash. The copilot was recently hired and still on probation. 
Foushee (1984) speculates that had the copilot not been intimidated, he would 
have assumed control and possibly averted the disaster. 

Communication is the process by which information is clearly and 
accurately exchanged between two or more team members in the prescribed 
manner and by using proper terminology, and the ability to clarify or 
acknowledge the receipt of information. 

Perhaps the most powerful index of team performance is the extent and 
character with which team members interact with each other. 
Communication underlies every team process discussed above. As a process, 
effective communication couples team members' expectations, actions, 
responses, and feedback behaviors. Communication facilitates the 
transmission and reception of support behavior, and the detection and 
correction of error conditions (Mclntyre & Salas, in press; Oser et al., 1989), 
helps team members synchronizes, their activities (Foushee, 1984; Kanki et ai., 
1989a, 1989b; Mclntyre & Salas, in press; Mullen & Copper, 1994), influences 
the quality of decision making (Brannick et al., 1993; Cannon-Bowers et al., 
1993; Rouse et al., 1992), influences the character of team cohesion (Mclntyre & 
Salas, in press; Oser et al., 1989), and establishes operational norms between 
members (Foushee et al, 1986; Mclntyre & Salas, in press; Mullen & Copper, 
1994). It should be noted that excessive communication is counterproductive 
(e.g., Lahey & Slough, 1982). 

Team Processes in a Negotiation Context 

The negotiation context within which we frame our teamwork processes 
is the mixed-motive interdependence paradigm (Bazerman, 1990; Carnevale & 
Pruitt, 1992; O'Neil et al., 1994). Briefly, this paradigm incorporates two 
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dimensions of negotiation styles, distributive and integrative. Distributive 
negotiation reflects a view that the issues under negotiation are equally valued 
by both parties; conceding on an issue necessarily means a loss. Such a view 
leads to a negotiating strategy of attempting to gain as much as possible across 
all issues— a zero-sum/win-lose perspective. Integrative negotiation reflects a 
view that the issues under negotiation have differing values for both parties. A 
concession on an issue does not necessarily mean a loss. Rather, conceding on 
an issue of low importance may result in gaining on an issue of high 
importance. This leads to a negotiation strategy of seeking ways in which both 
parties can benefit. Integrative ne^, rtiation reflects a variable-sum/win-win 
view of the situation. Expert negotiators adopt a integrative negotiation style. 
The mixed-motive interdependence paradigm has the potential to elicit many 
of the team processes defined above. In the following discussion we analyze 
each team process in the context of this negotiation scenario, outlining the 
kinds of behaviors we expect to emerge. 

Adaptability. In general, a team's adaptive capability affects the nature of 
the team's response to a given situation. Are team members able to detect 
problems as they arise and, once detected, respond effectively to resolve the 
problems? Do team members exhibit innovative and flexible responses to 
situations? 

Applying our definition of adaptability to the negotiation scenario suggests 
two kinds of outcomes. First, members exhibiting adaptive behavior should 
show more willingness to be integrative in their responses. For example, 
flexible team members would recognize that not all issues are equally 
important and would thus carry out negotiations accordingly. Issues of low 
importance could be traded away for gains on issues of high importance. 
Conversely, members showing little adaptability might not recognize this 
situation and would hold firm across all issues. We expect integrative 
behavkr to manifest itself as questions or statements that try to uncover the 
other party's priorities and values. 

A second outcome of adaptive behavior is the use of compensatory 
feedback. That is, members provide information to each other regarding the 
current situation. We would expect compensatory behavior to manifest itself 
as members agreeing, disagreeing, asking for ideas, or suggesting possible 
courses of action. 
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Coordination. In general, a team's coordination capability affects how 
that team organizes its resources, activities, and responses. Teams with high 
coordination will carry out a task that is integrated, synchronized, and 
completed on time. 

Coordination in the context of a negotiation scenario suggests that 
sequencing would be important. We believe that members of coordinated 
teams will show a propensity to sequence offers in an attempt to tease out the 
other team's values and priorities. Conversely, members of less coordinated 
teams will show a tendency to propose offers that will vary widely from round 
to round. In addition to sequencing, we also expect that highly coordinated 
teams will be cognizant of time constraints and communicate this aspect to 
other members when appropriate. 

Decision making. A team's decision-making capability affects its ability 
to capitalize on available information. Effective teams use information about 
their current situation to help them evaluate the worthiness of potential 
courses of action. 

In the context of a negotiation scenario, we expect to see several kinds of 
decision-making patterns contingent on how weil the teams understand the 
negotiation situation. First, for teams that enter the negotiation simulation 
with an existing understanding of how to negotiate, we expect that these teams 
will quickly reach a resolution and engage in little debate over what should be 
the proper strategy. In effect, these teams have shared mental models of the 
negotiation situation, and this understanding helps the team reach an 
agreement. 

For teams with less of an understanding (e.g., novices), one of two things 
c-mld happen. First, these teams could come to an understanding by reasoned 
discussion. Such teams would pursue a course of action after discussing 
alternatives and potential consequences. Thus, we would expect members to 
provide specific reasons for a particular position, and that these reasons reflect 
an integration of information about the specific negotiation situation. For 
example, a member might suggest going lower on an issue because that issue 
is important to the other party. Alternatively, teams with less of an 
understanding could simply engage in help-seeking behavior. That is, 
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communication between members would be dominated by appeals for ideas 
about what to offer, what tactic to take, and the like. 

Interpersonal. In general, a team's interpersonal capability affects how 
well team members work cooperatively. Do team members encourage each 
other to work together? Do team members help resolve team member dissent? 

In terms of our negotiation, scenario, we would expect teams with a high 
interpersonal dimension to encourage member participation, help seeking, 
and help giving. In addition, we would expect members to freely exchange 
compliments. 

Leadership. In general, a team's leadership capability affects its ability to 
provide direction for the team. To what extent does a member coordinate the 
activities of other members? Does the team engage in planning and 
organization that reflect appropriate priorities? 

In our negotiation scenario we expect that teams or team members high 
in leadership will engage in communication that provides direction and sets 
priorities. We would expect a member strong in leadership to place the 
current negotiation situation in context of the team goal. For example, such a 
member would remind other members of the team's priorities, thereby 
aligning task engagement with task goals. We also suspect that strong 
leadership would manifest itself in directives. In this case, communication 
would take the form of directing others to do something. Finally, we believe 
that good leadership will be reflected in a leader who is receptive to input from 
other members. 

We believe that our earlier efforts using an individual-level negotiation 
scenario (O'Neil et al., 1994) provide a useful context for studying team-level 
processes. In the next section we first discuss current approaches to 
measuring teamwork processes and their limitations. We then describe how 
we address those limitations in our research. 

Measuring Teamwork Processes 

Our work focuses on assessing team processes that emerge during the 
negotiating of a contract. We are interested in the nature of the interaction 
between team members and how that interaction impacts team performance. 
Yet a critical measurement issue remains unresolved: How do we assess 
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teamwork processes such that the measurement technique is reliable, valid 
and timely? 

Our previous work on using computer simulation suggests that our 
approach is feasible, reliable, and valid when measuring individual 
negotiation skills (O'Neil, Allred, & Dennis, 1992; O'Neil et aL, 1994). Our 
simulation technique reliably discriminated between experts and novices, 
providing strong support for the validity of our approach. Our findings 
persisted across different negotiation contexts and populations (i.e., high 
school students negotiating for a job at a movie theater, law students 
negotiating for a job with a law firm). 

However, our preliminary attempts at using teams in the same simulated 
negotiation context were inconclusive (O'Neil et al., 1994). In O'Neil et al. 
(1994, Study 2), 3-person teams were compared to 2-person teams. The 
negotiation scenario was identical to that whicl individuals engaged in. The 
primary difference between the scenarios was that the team scenario req aired 
the team members to agree verbally on a course of action. Then, one team 
member would communicate that decision to the computer. 

Although O'Neil et al. (1994) found nonsignificant effects on one measure 
of performance (final counteroffer), the results were in the predicted direction 
(3-person teams performing better than 2-person teams). On the measure of 
quality of performance (integrative vs. distributive), O'Neil et al. found no 
significant differences. Finally, in terms of frequency of agreement, O'Neil et 
al. did find significant effects. Three-person teams tended to reach agreement 
more often than 2-person teams. While the measures used in O'Neil et al. 
(1994) captured team outcomes, they did not capture the underlying teamwork 
processes. Thus, the focus of our current work is to develop methods for 
measuring teamwork processes. 

Existing approaches to measuring teamwork processes rely almost 
exclusively on observational methods (Baker & Salas, 1992). For example, 
behavioral checklists (e.g., Oser et al., 1989), videotaped and audiotaped 
observation (e.g., Brannick et al., 1993), and analysis of think-aloud protocols 
are the most common techniques to measure teamwork processes. These 
methods are labor intensive and time-consuming. Observations must be 
transcribed, coded, and analyzed post hoc. Such techniques offer no 
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opportunity for rapid analysis and reporting of team performance. From an 
assessment perspective, these methods are unappealing becav.se of the lag 
between test administration and reporting of test results. Further, these 
methods are neither practical nor cost-effective in large-scale test settings. 

Our approach to addressing these limitations is to expand our computer- 
based negotiation simulation to a new context (union-management negotiation 
of a contract) and to incorporate real-time measures of teamwork processes. 
To achieve this, we broadened the scope of the computer-based negotiation 
simulation in the following way. Individuals were assigned to their own 
computers and collaborated with team members through a custom-developed 
computer conferencing system (e.g., McCarthy, Miles, Monk, Harrison, Dix, 
& Wright, 1993). Note that our initial system implemented the sending of both 
predefined messages and free-form messages; more typical conferencing 
systems allow for sending of typed-in, free-form messages only. This approach 
of linking team members via a network differed from our preliminary 
investigation of teams (see Study 2 of O'Neil et al., 1994 for differences). In that 
study, O'Neil et al. (1994) physically grouped one team with one computer (e.g., 
three participants sharing one omputer). Discussion took place face-to-face 
but was unrecorded. In the pre. >nt study, each individual operated his or her 
own computer and interacted with other team members by sending messages. 

Our final technique for capturing teamwork processes is to use a pre- 
defined taxonomy of teamwork skills discussed earlier and categorize the 
expected communication acts team members would engage in. We have 
developed a priori a set of "canned" messages which members can send to each 
other. Further, we have categorized these messages as belonging to one of the 
five teamwork processes (adaptability, coordination, decision making, 
interpersonal, and leadership). This taxonomy of teamwork is domain 
independent and independent of scenarios. By tracking the messages selected 
and sent (and hence, by definition, the teamwork process category) we get a 
good index of the kinds teamwork processes emerging. We assume that each 
message in a category is as important as any other message, and thus all 
messages are equally weighed. Using those techniques provides us with a 
real-time teamwork assessment system. We can administer, score and 
interpret in real time. The Methods section covers the messages and their 
associated categories in greater detail. 
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Pilot Studies 

Two pilot studies were conducted to assess the feasibility of our approach. 
The first pilot study represented our initial attempt at assessing the 
functionality of the computer system and messages. We were interested in 
feedback from users regarding usage of the system, messages, and task 
performance. The second pilot study reflected several major revisions to the 
system based on the first pilot study. The two pilot studies are described in 
greater detail below. 

Pilot Study 1 

Participants. The first pilot study used three groups of adults. 
Participants were drawn from available graduate, undergraduate, and other 
nonstudent adults in our laboratory. Two groups were run formally, where 
participants worked as a team under controlled conditions. In these groups, 
the programmer of the computer system participated and assumed the role of 
an advisory member. The third group was run informally, where participants 
provided comments about the system as they carried out the negotiation task. 

Negotiation task. For our pilot study we created a scenario where 
participants collaborated in 3-person teams. Each team consisted of one leader 
and two advisory members. The team assumed the role of representing a 
union, and the team's responsibility was to negotiate a contract with a 
fictitious (i.e., simulated) management team. The issues under negotiation 
were wages, health and welfare, and pension. The simulated management 
team was programmed to concede on its most important issue when the union 
(i.e., participant) team did likewise (O'Neil et al., 1994). Thus, the simulated 
management team mirrored the negotiating behavior of the participants' 
team. The team outcome measures, based on previous research (O'Neil et al., 
1994), were (a) whether agreement was reached and (b) whether the 
agreement was integrative or distributive. 

Messages. In order to capture team processes in real time, we developed 
a set of messages for each teamwork category (e.g., interpersonal skills) 
identified above. Traditionally, measurement of teamwork skills is done using 
human judges either to rate observations (live or videotaped) of a team 
interacting, or to rate transcripts of team members' think- aloud protocols. In 
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our case, we developed messages a priori to represent team processes. Our 
intent was to have team members use these messages to communicate with 
each other regarding the negotiation. For our feasibility study, we included a 
variety of message types that reflected good and poor team skills. The 
following list gives examples for each category: (a) adaptability — "We should 
try something different"; (b) communication— "I don't understand, please 
explain"; (c) coordination — "We need to hurry up"; (d) decision making— "We 
might want to give a lot on pension"; (e) interpersonal — "I am interested in 
hearing from you on this"; (f) leadership— "Let's place an offer now"; and (g) 
motivation — "Our objective is to maximize our interests." 

Computer system. The computer platform we used was the Macintosh 
(1993), and the software was developed with HyperCard 2.2 (1993). We used the 
built-in networking capabilities of the operating system (System 7, 1992) and 
HyperCard (1993) to implement a rudimentary client-server system. We 
designed the computer system to enable members to communicate with each 
other via a common message blackboard. Messages were posted as they were 
dibpatched by members. All members had read and write access to the 
blackboard. To send a message, participants first selected one of the message 
categories. A list of predefined messages for that category appeared, and 
participants either chose a predefined message or entered their own message. 
The computer system recorded the number of messages (both predefined and 
self-generated) sent from each category. 

The message sending capability was organized around teamwork 
category. To send a message, participants first clicked on a button 
representing one of the teamwork categories. Then, a window opened that 
presented a list of predefined messages for that category as well as an option 
where the participant could type in his or her own message. If the participant 
chose to send a predefined message, he or she simply clicked on the message 
and the message was dispatched. Alternatively, the participant could type in a 
message. Participants were also provided with a paper listing of all the 
messages sorted by teamwork category. 

Results. For the first group, the maximum number of rounds was set to 
12. (We define one round to consist of an offer from the union team and a 
counteroffer from management.) The total number of messages sent was 231, 
and no agreement was reached. The simulation ended after the allotted 12 
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rounds and ran for 100 minutes. For the second group, the maximum number 
of rounds was set to 10. The total number of messages sent was 124, and 
agreement was reached on round 8 after 82 minutes. For the third group, the 
maximum number of rounds was set to 8. The total number of messages sent 
was 49, and agreement was reached on round 7 after 70 minutes. These 
results of the first pilot study point to strengths and limitations of our initial 
approach. These findings are outlined below. 

All groups participated with sustained interest and effort. Despite 
participants' knowledge that they were negotiating with a simulated 
management team, all groups approached the task with concerted effort. 
Participants reported that the task was interesting and that they were 
genuinely trying to reach an agreement. 

A review of the message transcripts also suggests a consistent pattern of 
communication. In general, communication repeatedly took the form of a 3- 
stage process. In the first stage, someone (usually the leader) would suggest a 
specific offer. The second stage was characterized by a discussion of the pros 
and cons of that offer, how the offer might impact the team, and potential 
actions the management team might take in response to the offer. During this 
stage alternative offers were suggested by all team members. The last stage 
was characterized by someone (usually the leader) making a suggestion to 
wrap up the discussion and make a decision. The leaders of all groups sought 
consensus before sending the offer to management. There was no evidence of 
a leader acting independently or dictatorially. 

The networked computer simulation clearly evoked a strong desire to 
communicate. Participants wanted to communicate immediately, and using 
the enter-own-message facility was the most efficient way to do it. In the two 
formal groups, team members started using the predefined messages in 
earnest, but quickly resorted to the enter-own-message facility. In the third 
group, all members immediately resorted to and relied on the enter-own- 
message facility. 

All three groups avoided using the predefined messages primarily 
because they had difficulty interpreting the message categories. Participants 
uniformly reported that the message categories were unclear, ambiguous, and 
cumbersome to use. The presence of message categories impaired rather than 
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facilitated communication. Generating a message was simply easier than 
searching for an appropriate one. A review of the transcripts suggests the 
ease of entering messages was a major reason for not using the predefined 
categories and messages. However, for many of the self-generated messages, 
the substance of those messages mapped reasonably well onto our predefined 
message categories. 

Pilot Study 2 

The results of the first pilot study provided insight regarding our initial 
approach and informed us about needed revisions. First, although the 
presence of the enter-own-message facility clearly enhanced communication 
and was the preferred method of communication, the number of overlaps 
between the self-generated messages and the predefined messages suggested 
that we were able to capture the substance of what participants wanted to 
communicate. Second, the finding of a stable communication pattern 
suggested a different way of organizing the messages. Rather than grouping 
messages by teamwork categories, this finding suggested grouping messages 
by communication function. We were also able with this scheme to avoid 
teaching what the categories were and how to use them. 

Incorporating these findings in the second pilot study required two major 
changes to the system. The first change involved eliminating the enter-own- 
message facility. Our intent was to capture team processes in real time; 
maximizing ease of communication was of secondary importance. Results 
from the first pilot study suggested that our predefined messages were able to 
capture what participants wanted to say. In addition, we refined our message 
set to reflect what pilot study participants had typed in. 

The second change was made to the way the messages were organized for 
participants. In the first pilot study the organization was done on-screen (i.e., 
having one button per teamwork category). In the second pilot study we shifted 
the organization from the computer screen to a paper handout. The handout 
listed all the messages and was organized functionally. For example, 
messages that were commonly used were grouped together under the heading 
"Quick Responses." Each message on the handout corresponded to a button on 
the screen. To send a message, participants clicked on the button that 
corresponded to the desired message on their handout. 
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Participants and negotiation task. Six adults (two groups) were drawn 
from available undergraduates. Participants took part in the same negotiation 
task as participants in the first pilot study. 

Messages. For the first group, we revised the message set from the first 
pilot study, incorporating new messages based on what participants typed in. 
In addition, we generated additional messages that we believed to be consistent 
with our definitions of team categories. Participants received a handout with 
messages divided into eight areas. The areas were (a) quick responses, (b) 
input from other members, (c) negotiating priorities, (d) entering offers, (e) 
discussing offers, (f) wages, (g) health and welfare, and (h) pension. The 
handout listed 86 messages. 

For the second group, we used feedback from the first group to reorganize 
the handout. We included additional messages that they suggested, as well as 
new messages we generated. For this group, we partitioned the handout into 
11 sections. These sections were (a) a flow-chart depicting the 3-stage process 
we observed during the first pilot study, (b) quick responses, (c) seeking 
assistance, (d) team spirit, (e) entering offers, (f) adjusting offers, (g) reasons 
for going higher/lower on each issue, (h) time concerns, (i) sending offers, (j) 
awareness of our team's priorities, and (k) awareness of the management 
team's performance. 

Results. For the first group, the maximum number of negotiation rounds 
was set to 8. The total number of messages sent was 270, and agreement was 
reached on round 8 after 60 minutes. For the second group, the maximum 
number of rounds was also set to 8. The total number of messages sent was 
193, and agreement was reached on round 6 after 55 minutes. 

Postsession interviews with groups revealed that members felt the system 
was usable. Members said they were able to communicate with each other 
adequately (although all participants wanted the capability to enter their own 
messages). Participants found the message set to be, on the whole, adequate to 
carry out the negotiation task. 
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Hypotheses 

Drawing on the taxonomy of teamwork processes outlined bv Burke et al. 
(1993) and our analyses of these processes in the context of the mixed-motive 
interdependence negotiation, the following hypotheses are suggested. 

Hypothesis 1: Team processes will I - positively associated with team 
performance and time. 

We expect that teams that exhibit higher levels of the team processes 
(adaptability, coordination, decision making, interpersonal, and leadership) 
will perform better on the negotiation simulation. Performance is defined in 
our scenario in four ways: (a) final counteroffer, (b) frequency of agreement, 
(c) type of agreement (integrated vs. distributed), and (d) time. The widespread 
finding that effective teams engage in these kinds of processes leads us to 
believe that team performance on our negotiation simulation will reflect 
similar patterns. 

Hypothesis 2: Team performance will be negatively associated with time-to- 
agreement. 

As suggested by the results of Brannick et al. (1993) and the work on 
shared mental models (Cannon-Bowers et al., 1993; Rouse et al., 1992), we 
expect time to be sensitive to team decision making. Specifically, we expect 
that teams engaging in more communication will take longer to reach an 
agreement. 

Hypothesis 3: Team performance will be negatively related to the degree of 
fixed-pie bias. 

Our prior results (O'Neil et al., 1994) indicate that good performers 
(experts) displayed less fixed-pie bias. 

Hypothesis 4: More self-regulatory activity will result in higher team 
performance. 

Our prior results (O'Neil et al., 1994) suggested that more self-regulating 
activity and less worry distinguished experts from novices. Thus, we expect 
dat higher self-regulating activity would lead to higher team processes and, 
in turn, better team outcomes. 
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Participants 

Eighty-one participants (34 male, 47 female) participated in this study. 
All participants were drawn from a high school located in southern 
California. One teacher provided four classes of students. Studer s 
participated as part of that day's class activity. UCLA and USC procedures for 
protection of human subjects were followed (e.g. parental approval, informed 
consent). 

The overall ethnic composition of the paiticipants was 29 White/Anglos, 3 
Black/African Americans, 4 Hispanic/Latinos, 38 Asians, and 7 Middle 
Eastern/Persians. Participants' ages ranged from 14 to 18 years old (60 
sophomores, 21 seniors). Participants came from three lOth-grade classes (18, 
18, and 24 participants respectively) and one 12th-grade class (21 participants). 
The content of all classes was American history. 

Participants drawn from these intact classes covered a range of academic 
standing, ethnic, and gender compositions. The first class was of normal 
academic standing (sophomores) and was composed primarily of Asian, 
Middle Eastern/Persian, and female students (2 Whites, 2 African Americans, 
1 Hispanic/Latino, 9 Asians, 4 Middle Eastern/Persians; 6 males, 12 females). 
The second class was a lOth-grade "resource class" (each student had an 
individual educational plan). This class was composed primarily of White 
students (11 Whites, 2 Hispanic/Latinos, 3 Asians, 2 Middle Eastern/Persians; 
9 males, 9 females). The third class was of normal academic standing 
(seniors) and composed primarily of White students (14 Whites, 1 
Hispanic/Latino, 5 Asians, 1 Middle Eastern; 11 mahs, 10 females). The 
fourth class was a lOth-grade Advanced Placement class composed primarily 
of Asian and female students (2 Whites, 1 African American, 21 Asians; 8 
males, 16 females). 

Negotiation Scenario 

The negotiation aspect of the task followed the format established in 
previous studies (e.g., O'Neil et ah, 1994). As discussed earlier, the present 
study extended the negotiation task by adding a teamwork dimension. Table 1 
presents the general domain specifications embedded in the software. 
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Table 1 




Domain Specifications Embedded 


in the Software 


General domain specification 


OpcClIlL cXdnipic 


Scenario 


Role play a contract negotiation by exchanging proposals 




in mi YPfl-mntivp pnntPYt" 

111 1111 ACU lli L' H V C \sVJll l/CA l» 


Players 


One or more students and one manager (computer 




software) 


Student 


Either expert or novice, individual or team 


Manager 


Computer software (Carnevale & Conlon, 1988; O'Neil, 


Allred, & Dennis, 1992) 


Priorities 


Offsetting 


Moves 


Reciprocal 


Rounds 


Offer from student and counteroffer from manager 


Subcompetencies 


Propose options; make reasonable compromises 


Negotiation issues 


Three in number (e.g., salary) with offsetting priorities 


Negotiation measures 


Agreement (yes/no ',, type of agreement (distributive vs. 




integrative), final counteroffer 


Cognitive processes 


Fixed-pie bias 


(domain-dependent) 




Cognitive processes 


Metacognitive skills 


(domain-independent) 




Affective processes 


Effort, Worry 


(domain-independent) 





The scenario presented to participants was that of a union-management 
labor negotiation situation. The ecological validity of our scenario was 
facilitated by information from the union and management leadership of the 
Southern California Airconditioning and Refrigeration Industry. Values for 
the negotiation issues were used from the industry's Master Agreement 
(Southern California Airconditioning and Refrigeration Industry, 1990). Our 
messages were also reviewed by the industry and union leadership (see 
Table 2). In general, they felt that we had captured the formal negotiation 
process but that the informal process was not well captured (e.g., business 
conducted on the golf course, the use negative interpersonal remarks). 
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Table 2 

Team Work Skills 



Adaptability 

- Recognizing problems and responding appropriately 

If we give on wages, they might give on pension. 

Coordination 

- Organizing team activities to complete a task on time 

We need to hurry to complete this round. 

Decision making 

- Using available information to make decisions 

We should go higher on liealth and welfare because 
it's their second priority. 

Interpersonal 

- Interacting cooperatively with other team members 

We are doing a great job. 

Leadership 

- Providing direction for the team 

Send the offer. 

Communication 

- Information is clearly and accurately exchanged between 
team members 

The total of messages minus "I sent the wrong 
message. " 



Participants were informed that they were part of a 3-person team 
representing the union or management (in fact, all teams were representing 
the union; the computer simulated the management team). Participants were 
informed that their team's responsibility was to reach an agreement with the 
other side, and that the team should work together on what to offer the other 
team. Participants were informed they had eight rounds to complete the 
negotiations, and they were not told who their teammates were. Further, the 
negotiation scenario was not an interdependent task. It could be solved by a 
single student. Thus, in our scenario, the task is very similar to a 
collaborative learning math environment in K-12. 
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The negotiation was terminated under the following conditions. First, the 
team could accept the management team's offer at any time. Second, if the 
team sent an offer that was acceptable to the management team, then the 
management team would accept that offer. An offer that was integrative in 
nature (over 120 points) was accepted. The last way the negotiation could end 
is if no agreement was reached within the allotted 8 rounds. In that case, a 
final counteroffer was generated by the software. 

Tables 3, 4 and 5 give examples of each case. Table 3 shows a group 
accepting the management team's offer at round 6. Table 4 shows the group's 
offer being accepted by the management team at round 8. Finally, Table 5 
shows the group not reaching an agreement within 8 rounds. 

Negotiation issues and priorities. The issues under negotiation were (a) 
wages, (b) health and welfare, and (c) pension. Participants were informed 
that their team's priorities were (in order of importance) wages, health and 
welfare, and pension. 

To help participants gauge the value of the offers during negotiation, they 
were provided with an "issue chart." The issue chart was part of the computer 
display and hence always available. Participants were instructed to use these 
values to determine the value of their offer to the management team. 
Participants were instructed that they should try to negotiate an agreement 
that optimizes their total number of points. Table 6 shows the issue chart for 
the participants' team, and Table 7 shows the issue chart for the simulated 



Table 3 

Sample Negotiation With the Union Team Accepting the Management Team's 
Offer 



Round 


Management (computer) offer 


Union (participant) offer 


1 


15-10-5 


120-80-35 


2 


15-0-0 


120-70-30 


3 


30-10-0 


105-70-25 


4 


45-10-5 


90-60-20 


5 


60-20-10 


75-50-20 


6 


60-30-15 


The union team accepts the 






management team's offer. 



Note. The management (computer) team always mnkes the first offer. 
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Table 4 

Sample Negotiation With the Management Team Accepting the Union 
Team's Offer 



Round 



Management (computer) offer 



Union (participant) offer 



1 

2 
3 
4 
5 
6 
7 
8 



15-10-0 

15-20-5 

15-20-10 

45-30-10 

75-30-20 

45-40-15 

45-40-20 

The management team accepts 
the union team's offer. 



105-60-35 
90-60-35 
90-50-25 
60-50-25 
75-40-25 
60-40-15 
60-40-20 



Table 5 

Sample Negotiation With No Agreement Reached 



Round Management (computer) offer Union (participant) offer 



1 


15-10-5 


105-80-35 


2 


15-0-5 


75-60-35 


3 


15-20-15 


90-70-35 


4 


15-10-10 


90-70-35 


5 


15-10-10 


60-50-35 


6 


105-30-20 


105-40-35 


7 


15-40-5 


105-30-35 


8 


15-30-5 


75-40-25 



management team. Note that the issue priorities for the simulated 
management team are the reciprocal of the union team's priorities. 
Participants were not informed of the simulated management team's 
priorities. 

The simulated management team (computer) was programmed to reflect 
priorities that were opposite to the union's team (see Appendix 1 for the 
computer algorithm). That is, the computer's highest priority was Pension 
followed by Health and Welfare and Wages (see Table 7). The computer's 
responses were programmed to reflect an integrative potential. If the union 
team gave on its value of least importance (Pension), the computer would give 
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Table 6 

Issue Chart for the Union (Participant) Team 



Wages Health and Welfare Pension 



120 


80 


40 


105 


70 


35 


90 


60 


30 


75 


50 


25 


60 


40 


20 


45 


30 


15 


30 


20 


10 


15 


10 


5 


0 


0 


0 



Table 7 

Issue Chart for the Management (Computer) Team 



Wages 


Health and Welfare 


Pension 


0 


0 


0 


5 


10 


15 


10 


20 


30 


15 


30 


45 


20 


40 


60 


25 


50 


75 


30 


60 


90 


35 


70 


105 


40 


80 


120 



on its issue of least importance (Wages), and the joint outcome would be 
maximized. Further, the computer was programmed to make offers that 
mirrored the union team's offers. If the union team gave on its most 
important issue (Wages), the computer would give on its issue of most 
importance (Pension). For the Health and Welfare issue (moderate priority to 
both the union and management team), the computer responded with a tit-for- 
tat strategy: A union concession would be mirrored by a equal concession by 
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the computer. In addition, the computer would accept any union offer that 
totaled 120 points or more (when computed from management's point of view). 
Thus, in some cases there was an implicit ceiling on the final counteroffer 
measured. As previously mentioned, the scenario was adopted from O'Neil et 
al. (1994). Table 8 shows the comparisons between the two versions. 



Table 8 

Comparisons Between the Current Software and That Used in O'Neil et al.'s (1994) Study 



Item 


Macintosh (current) 


IBM (O'Neil et al., 1994) 


User interface 


Graphical user interface; user selected values and 
options via dialog boxes, pull-down menus, and 
clicking on buttons. Only black-and-white used. 


Text display, user typed in 
offer values. Color used to 
emphasize information. 


Offer processing 


Accepting an offer done by computer and user. 


Accepting an offer is 
incumbent on user. The 
computer NEVER accepts an 
offer. 


No. of rounds 


o 
o 


19 


Raw data 
processing 


Data is time-tagged and indexed with embedded 
comments. Redundant data archiving — data 
saved to text file and HyperCard stack. Program 
State savea as v/eii. 

Raw data fully recoverable from HyperCard stack. 


Single stream of data saved 
to a text file. 


Development 
soltware 


HyperCard 2.2 


Turbo Pascal 4.0. 


Networking 


3-person network in client-server configuration. 
Use AppleEvents to pass data between computers. 


No networking. 


Instructions 


Delivered as a computer tutorial with 
demonstrations. 


Delivered by 
experimenters. 


Issue chart 


Displayed online with current offer highlighted. 


Displayed online. 


Job aid 


Handout that listed available messages. 


None. 


Amount of help 


Two experimenters plus classroom teacher. Help 
given only explaining how to operate computer and 
explain terms. No help given on how to reach an 
agreement or type of agreement. 


Two experimenters. No 
classroom teacher. 
Otherwise, same. 


Required 
computer skills 


Typing helpful, mouse skills. 


Typing helpful. 


Post processing. 


Raw data imported to relational database; data 
available for post-processing. Because data are 
indexed, able to selectively pull data for subsequent 
analysis. 


None. 


Message database 


Messages tracked in relational database. Message 
revision history tracked; comments, pilot data 
usage tracked; interrater reliability tracked; 
development comments tracked. 


Not applicable. 
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Computer Conferencing System 

Three major components comprised the system: (a) predefined messages, 
(b) a handout that listed all messages, and (c) software to support the computer 
conferencing system. Each component is described below. 

Messages. In order to capture team processes in real time, we developed 
a set of messages for the five teamwork categories. We developed a variety of 
messages a priori. Team members selected these messages to communicate 
with each other during the negotiation. 

FL'ty-one messages were provided to participants for use during the 
negotiation task. Fifty messages (10 messages/category) belonged to the five 
teamwork process categories discussed earlier (adaptability, coordination, 
decision making, interpersonal, and leadership). One message was provided 
to signal that a previously sent message was mistakenly sent by the 
participant. This message was excluded from th3 message categories. A 
sample message from each category is given below. Appendix 2 contains a 
complete list of messages and their categories. 

Adaptability — recognizing problems and responding appropriately. 
A message belonging to this category is "What do you think, Ml?" 
(Ml refers to Member 1 of the team.) 

Coordination — organizing team activities to complete a task on time. 
A message belonging to this category is "We only have 5 rounds left." 

Decision making — using available information make decisions. A 
message belonging to this category is "We should go higher on Wages 
because it's OUR 1st priority." 

Interpersonal — interacting cooperatively with other team members. 
A message belonging to this category is "Help me out." 

Leadership — providing direction for the team. A message belonging 
to this category is "Send the offer." 

Communication — the overall activity of the teams. No messages 
were tied to this category; rather, communication represented the 
aggregate interaction between team members minus the number of 
error messages. Our definition of communication was that accurate 
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and timely messages were sent. Because all of the messages were 
selected and accurately sent via the network, we considered all of the 
team process messages were examples of communication. 

Message handout. Participants received a paper handout with all 51 
messages. The organization of the handout followed a 3-stage model of 
communication patterns that emerged during pilot testing. During the first 
stage, someone (usually the leader) would suggest a specific offer. The second 
stage was characterized by a discussion of the pros and cons of that offer, how 
the offer might impact the team, and potential actions the management team 
might take in response to the offer. During this stage alternative offers were 
suggested by all team members. The last stage was characterized by someone 
(usually the leader) making a suggestion to wrap up the discussion and make 
a decision. Figure 1 shows how the handout was partitioned. (See Appendix 3 
for the actual handout.) 

As Figure 1 shows, the top section provided general instructions on how to 
use the handout. This section also contained a flow chart that depicted the 3- 
stage process. The middle and bottom sections contained the actual messages. 
The middle section contained messages that were unique to each stage, and 
the bottom section contained messages that were common to all stages. (See 
also Appendix 3.) 



Instructions, Flow Chart 


Entering 


Discussing 


Sending 


Offers 


Offers 


Offers 


Common Messages 



Figure 1. Layout of message handout used by participants 
during the negotiation task. 
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The "Discussing Offers" area was subdivided by function into five sections: 
(a) entering offers, (b) adjusting offers, (c) assessing performance, (d) keeping 
track of time, and (e) going higher/lower on the issues. Similarly, the 
"Common Messages" area was subdivided into two sections: (a) encouraging 
others and (b) quick responses. 

Hardware/Software 

The school's computer lab was used to run this study. The lab contained 
18 computers, and we provided 6 additional computers. The 18 lab computers 
were Macintosh LC475s and the remaining computers were 5 PowerBooks and 
a SE/30. The 18 LC475s were networked with an Ethernet local-area-network 
(LAN), and the PowerBooks and SE/30 were networked with a LocalTalk LAN. 
The tutorial used to deliver task instructions was developed with MediaTracks 
(1990), and the software used in the negotiation task was developed with 
HyperCard 2.2 (1993). The computer operating system used was System 7 
(1992), and each computer had at least 4MB of RAM and 40MB of hard disk 
space. The computer conferencing software tapped the built-in networking 
and interprocess communication capabilities of HyperCard 2.2 and System 7 to 
implement message sending between computers. 

The user interface is shown in Figure 2. The display was partitioned into 
three major sections. The top-left section contained numbered message 
buttons. Each button corresponded to a particular message on participants' 
message handout. To send a message, participants clicked on a button and the 
message corresponding to that button would be sent to the other team 
members' computers. (See Appendix 3 for the message handout participants 
used to map messages to message numbers.) The lower left section of the 
screen displayed the messages sent by all the members. The messages were 
listed in the order sent. The right-hand section displayed the issue chart, 
current union and management offers, and (if the participant was the leader) 
buttons to send and accept offers. The team's current offer was selected by 
clicking on one of the issue values (highlighted numbers). The selection would 
appear on every team member's computer. Only the leader could select, send, 
or accept offers. 

For certain messages, user input was required. These messages were 
handled by the use of dialogs that mostly required simple point-and-clicking. 
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SELECT MESSAGE NUMBER 



12 3 4 



5 6 7 8 9 10 



11 12 13 14 IS 



16 17 18 19K"]21 22 23 24 1 



25 26 27 28 29 30 31 32 



35 34 35 | 



36 37 38 39 



MESSAGES, 



02-0-001 9/25/94 10:00:32 PM 



— Th* minig*m*nt t*im his propos*d 15-10-5. 
LD: Any id*is? (46) 

LD:Goodid*i. (47) 
LD: Whit do you think, M2? (44) 
LD: Whit if w« of far 105-60-20? (1) 
LD : W* should offer 60 on H&W. (4) 
LD: Our 2nd priority is H&W. (20) 

— Our off*r of 120-70-40 w«s s*nt to th* minig*m*nt t*im. 

— Wilting for i r*spons* from th* manig*m*nt team. 

— You will b* notifi*d vh*n th* minigm*nt t*im his mid* in 
offer. 

— Th* minig*m*nt t*im his propos*d 0-10-0. 



Wi§*s 



Hlth& 
W*lfir* 



P*nsion I 



C»1»«i Offer-- 



80 



CD 



Us: 



105 
90 
75 
60 
45 

on 
15 
0 



60 
50 
40 
30 
20 
10 
0 



70 



10 



40 



Round: [~~2~1 of 8. 



Send New Offer 



flccept Their Offer 



Figure 2. User interface for the team leader. Advisory team members' 
screens did not have the send and accept offer buttons. Note the three major 
sections (counter-clockwise from the top-left): message buttons, message 
display, and offer information. 



For example , Figure 3 shows how the message "What if we offer X-Y-Z?" was 
handled. In this case, the participant simply clicks on the desired issue and 
selects the desired value. 



Our Current Offer: — 
Mgt's Current Offer: 1 5-10-5 


(Wig*s) 


(H&W) 


(Pennon) 


LUhat if me offer! 105 ▼ 


✓ 80 
70 


40 ▼!? 






[msg# 1] [ c< 




OK t 




40 






30 






20 






10 






0 





Figure 3. Message requiring user input: selecting 
values. 
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A second type of message required users to type in a value, as shown in 
Figure 4. In this case, the value the participant typed in was checked against 
a list of acceptable values. If the value was not in the list, the user was notified 
and the value rejected. 

Measures 

The measures employed in this study focused on team level outcomes 
within a negotiation context, process measures of teamwork, metacognitive 
skills, fixed-pie bias perceptions, and manipulation checks. Each measure is 
described below. 

Team outcome measures. Our team outcome measures were based on 
the work of O'Neil et al. (1994) and were designed to measure four aspects of 
the negotiation scenario: (a) final counteroffer, (b) whether or not an 
agreement, was reached, (c) if an agreement was reached, the type of 
agreement (i.e., integrative or distributive), and (d) elapsed time on the 
negotiation task. 

The final counteroffer was defined as the sum of the offer levels across all 
three issues. If an agreement was reached between the management and 
union teams, the agreed-to offer was used to computer the final counteroffer. 
For example, if the agreed-to offer was 120-40-0, the final counteroffer was 160. 

If an agreement was not reached, the final counteroffer was computed 
from the offer the management team would have proposed given the last union 
offer. For example, suppose the last union offer was 120-80-40. In response to 



Ot'r Current Offer: — 
Mgt s Current Offer: 15-10-5 



[msg # 4] 



LUe should offer [60j on ✓UJages | . 

Pension \ 



Cancel 



OK 



5 



Figure 4. Message requiring user input: typing 
in values; selecting values. 
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this offer, the management team would counterpropose 0-0-0. Thus, the final 
counteroffer would be 0. 

Agreement was defined as whether or not the union team reached an 
agreement with the management team. No distinction was made for the 
direction of the agreement. That is, a union team accepting the management 
team's Oi p er was treated the same as the management team accepting the 
union team's offer. 

Type of agreement was defined as integrative or distributive. An 
integrative agreement was defined as the union team reaching an agreement 
worth more than 120 points (see Table 6). A distributive agreement was one 
where the union team reached an agreement worth 120 points or less. 

Elapsed time was defined as the time between the start of the negotiation 
and the termination of the negotiation. This time does not include the tutorial 
or question-answering activities. 

Individual and team-level teamwork process measures. The individual 
and team-level teamwork process measures were designed to capture the 
degree to which individuals and their team engaged in each of the team 
processes (i.e., adaptability, cooperation, decision making, interpersonal, 
leadership, and overall communication). Individual team process measures 
were computed by counting the number of messages sent in each team process 
category. For example, if a team member sent 10 messages in the adaptability 
category, that individual's adaptability score was 10. This method was used 
for all categories except communication. Communication was measured by 
taking the total number of messages sent by an individual less the number of 
"I sent the wrong message" messages. 

Team-level teamwork processes were measured by summing across 
individual category counts. For example, if all three team members sent 10 
messages in the adaptability category, then the team-level adaptability 
performance was 30. 

Self-regulation Questions 

These questions consisted of measures of metacognition (planning, self- 
assessment, awareness, and cognitive strategies). In addition, effort and 
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worry were also measured. The reliability and validity data are acceptable and 
can be found in O'Neil, Sugrue. Abedi, Baker, and Golan (1992). 

Fixed-pie Bias Questions 

Two fixed-pie bias questions, based on the work of O'Neil et al. (1994), were 
adopted to measure the degree to which participants viewed the negotiation 
context as either (a) a situation where the issues being negotiated are fixed, 
and that a gain on one side necessarily means a loss to the other side (a 
"win/lose" situation), or (b) a situation where the issues being negotiated are 
not equally important to both sides, thus creating a potential of gaining on an 
important issue by giving on a less important one (a "win/win" situation). 

Manipulation Checks 

Nine manipulation-check questions were designed to give us a sense of (a) 
how well participants understood their team's priorities, (b) how adequate 
participants found the predefined messages to be (as a communicative device), 
(c) whether participants knew the identity of their teammates, (d) how easy 
participants found the system to use, and (e) participants' experience with 
computers. Appendix 4 contains a list of the manipulation check questions. 
Finally, participants were given an opportunity to type in messages that they 
would have liked to have available. 

Procedure 

Each participant was randomly assigned to a computer, a team, and a 
role (leader vs. member). The experimenters introduced themselves, informed 
the participants that they were part of a study examining teamwork and 
negotiation skills, and gave a brief overview of the scenario and the activities to 
follow. All participants went through a negotiation tutorial on the computer. 
The tutorial explained in detail the negotiation scenario, the negotiation issues 
and priorities, the team scenario, and how to operate the computer. (Each of 
these was described earlier in this report.) 

Team scenario. Participants were informed that they were part of a 3- 
person team comprised of a leader and two members. The leader was the only 
one who could select offers for the union team, and send to and accept offers 
from the management team. The two members were responsible for advising 
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the leader. Participants were urged to r;ach consensus on what to offer the 
management team. 

Computer use. Participants were given computer demonstrations on how 
to carry out critical functions such as sending messages. This demonstration 
consisted of playing prerecorded animated sequences that, for example, 
showed the mouse moving to a button, clicking on it, and showing the 
computer's response. 

Participants who finished the computer tutorial were instructed to study 
the message handout while waiting for everyone else to finish. Once everyone 
finished the tutorial, participants began the negotiation cask. At the end of the 
negotiation task participants were given feedback regarding their 
performance. Participants were shown their individual performance score 
(number of messages sent for each message category), team performance 
(total number of messages sent by all team members for each message 
category), and negotiation performance (whether or not an agreement was 
reached), type of agreement (integrative or distributive if an agreement was 
reached, not applicable otherwise), and elapsed time for the negotiation task. 
Examples of the scoring and interpretation of the results to the participants 
are found in Appendix 5. 

Following the performance feedback, participants were presented with 34 
self-regulation questions, 2 fixed-pie bias questions, and 9 manipulation check 
questions. In addition, they had the opportunity to type in any messages they 
felt the system should have had. Appendix 5 contains the full text of the 
questions. Following the questions, the participants were presented with 
screens describing the purpose of the study, the nature of integrative ("win- 
win") and distributive ("win-lose") agreements, and the need for effective 
teamwork skills in today's workplace. Students were escorted back to their 
classroom, debriefed, and told that they were all union teams and that the 
management team was played by the computer. Any questions were then 
answered. 

Data Collection Problems 

The first attempt at data collection was marred by catastrophic computer 
failures. The class was comprised of 18 sophomores (6 groups of 3 
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participants). Throughout the initial data collection session computers would 
crash or freeze. Only one group out of the six completed the entire 
experimental session. The individuals in this group participated under 
standard conditions in a new group the following week. Thus, the majority of 
participants in the first class were not exposed to the negotiation task or 
subsequent questionnaire-type data collection. These students participated 
under standard conditions in new groups the following week. The remaining 
three classes (which did not participate in this first attempted data collection) 
were rescheduled for the following week. 

The computer failures were caused by an interaction of a utility used to 
extend the HyperCard (1993) program's functionality and the network 
functions of HyperCard. This utility (called a "XCMD"), which provided the 
window dialog capability (e.g., see Figures XX-1 and XX-2), also provided a 
special kind of window that could be programmatically opened and closed 
without user intervention (hereafter called a status window). We used status 
windows to notify the participant that the computer was engaged in a lengthy 
process. The status window also signaled to the participant that he or she 
should wait. Immediately after the process was finished the window was 
closed, signaling to the participant that he or she could continue. 

However, use of the status window caused the computer to crash when 
the following conditions occurred simultaneously: (a) the status window was 
being closed and (b) a message was received from another computer. The 
problem was eliminated by not using status windows. 

A second data collection attempt was made and completed the following 
week. Three computer failures occurred, of which only one failure was 
recoverable. During the first sessions one computer failed during the 
negotiation task. Members of this team were immediately escorted back to 
class and informed that the computer had failed. (These participants were 
dropped from our analysis.) The failure appeared to be similar to the one that 
occurred the previous week; however, we were unable to determine the exact 
cause of failure. During the second session, one computer failed after the 
participant completed the tutorial. Because there were spare computers, 
members of this team were moved to those computers. This incident occurred 
before the class started the negotiation task. The cause of this problem was a 
HyperCard system crash. This problem did not reoccur. The third computer 
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failure occurred in the third session during the negotiation task. This team 
was immediately escorted back to class. As with the first session, this failure 
appeared to be similar to the one that occurred the previous week, but its cause 
is unknown. 

Another incident occurred in the final session — the lOth-grade resource 
class (18 participants or 6 groups). This session was far noisier and rowdier 
than the other three sessions. In particular, some of the teams that finished 
the negotiation task later in the session were talkative and disruptive while 
answering the questions. One factor that contributed to the problem was that 
this group ran nearly to the end of the class period, which was the last class of 
the school day. 

Results 

Message Selection Procedure 

All of the messages provided in the negotiation scenario were selected 
from a much larger pool of potential messages derived from student messages 
in our pilot testing or developed by the authors to represent teamwork 
processes mentioned previously. The messages were sorted by two 
independent raters into five categories: adaptability, coordination, decision 
making, leadership, and interpersonal teamwork processes. Messages could 
be put in more than one category, with the condition that if a message was 
sorted into more than one category of team processes, such categories would be 
ordered according to the rater's perceived strength of fit for that message to 
each category. Thus, if Message 1 was sorted into both the leadership category 
and the adaptability category, leadership and adaptability would be ranked for 
this item according to which category the item fit into better, with 1 indicating 
the best fitting (primary) category, and 2 indicating the next best fitting 
(secondary) category. Almost all items were placed in two or fewer categories 
by each rater. 

From this larger pool of potential messages, items were selected which 
showed strong agreement between the two raters for each category, with the 
object being *c r«cain 10 messages in each category. Agreement between the 
raters occurred when both raters sorted an item into the same primary 
category. To obtain the desired quantity of messages in each category, 
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messages with secondary agreement were retained for those categories 
without ten messages with primary agreement. This practice provided an 
adequate number of messages for the leadership category. However, in the 
adaptability, coordination, decision making, and interpersonal teamwork 
process categories, one message was needed to complete the desired quantity 
for each category. These were determined by a third rater, who then rated 
those messages for which the first two raters failed to reach agreement. 
Thus, at least nine out of ten messages in each category (90% to 100%) resulted 
from primary or secondary agreement between the two initial raters. In 
addition, a single error message that did not fit into any of the teamwork 
process categories was included. The rater agreement for the five teamwork 
process categories is shown in Table 9. 

Individual-Level Measures 

Preliminary analysis of the data revealed that each of the 51 messages 
provided in the negotiation simulation were used at some point. The observed 
frequency for each message is provided in Table 10. As may be seen in 
Table 10, though all messages were utilized, individual message usage varied 
from 3 (message #26) to 551 (message #33). The frequency distributions were 
highly skewed positively, with most items having a mode of zero. Thus, prior 
to combining the items to generate team process scales, the message counts 
were transformed from raw score frequency of usage counts for each subject to 
percent of overall item usage for that subject relative to all subjects. That is, if 
Subject A had a frequency count of 6 on item X that had a total usage of 50, 
Subject A's score on item X was transformed from 6 to .12 (6/50). This put each 
item on a common metric (0-1) and preserved the underlying distribution 
because each item was merely transformed by division of a constant value. 

As was mentioned earlier, during the initial class periods of data 
collection, software system malfunctions made it necessary to reschedule 
subjects to the following week, once the malfunction problems had been mostly 
resolved. This resulted in one of the classes having some prior exposure to the 
negotiation software, since the malfunctions occurred after subjects had been 
introduced to the system and navigated through the tutorial, but before they 
started the negotiation scenario. 
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Table 9 

Rater Agreement for Teamwork Process Categories 



Teamwork process 


Primary 
agreement 


Secondary 
agreement 


Cumulative 
agreement for 
both categories 


Adaptability 


90% 


0% 


90% 


Coordination 


40% 


50% 


90% 


Decision making 


90% 


0% 


90% 


Leadership 


100% 


0% 


100% 


Interpersonal 


90% 


0% 


90% 



Table 10 

Frequency Count of Messages 



Message # 


Frequency 


Message # 


Frequency 


Message # 


Frequency 


1 


302 


18 


35 


36 


65 


2 


287 


19 


9 


37 


22 


3 


49 


20 


62 


30 


29 


4 


53 


21 


14 


39 


26 


5 


52 


22 


26 


40 


89 


6 


20 


23 


19 


41 


37 


7 


31 


24 


17 


42 


357 


8 


32 


25 


62 


43 


270 


9 


17 


26 


3 


44 


216 


10 


40 


27 


16 


45 


106 


11 


24 


28 


20 


46 


95 


12 


44 


29 


40 


47 


326 


13 


45 


30 


14 


48 


40 


14 


35 


31 


21 


49 


28 


15 


65 


32 


49 


50 


65 


16 


16 


33 


551 


51 


118 


17 


26 


34 


129 







Separate statistical analyses were conducted for those subjects who were 
not exposed previously to the program, for those who were, and for the group 
as a whole. The first session had 6 groups with varying levels of prior 
exposure to the program. The other three sessions had a total of 20 groups used 
in the analysis. Comparing those groups with exposure (iV=6) to those without 
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exposure (AT=20) on each of the outcome measures — score (M = 115, SD = 16.8 
vs. M = 124, SD = 32.7); time (M = 26.5, SD = 9.0 vs. M = 28.9, SD = 9.8); 
agreement (83% vs. 85%); agreement type (both at exactly 50% integrative, 50% 
distributive) — did not result in significantly different scores. Similar results 
were found for comparisons on all team process categories and self-regulation 
measures. No differences between the groups were found. In summary, 
separate analyses did not result in any substantive differences in performance 
on any of the measures or conclusions drawn therefrom. Thus, data from all 
four class sessions were retained for analysis and are reported here. 

For each subject, individual scores were calculated for the adaptability, 
coordination, decision making, leadership, and interpersonal teamwork 
processes by counting the messages from each category (see Appendix 2 for 
messages in each category). In addition, a total communication score was 
computed by summing each of the five teamwork process scores for each 
individual less the total number of error messages for that individual. An 
individual's scale score was not adjusted for error messages. Means and 
standard deviations for each scale are presented in Table 11. 

As may be seen in Table 11 of the observed frequency for each process, the 
average participant "sent" approximately 10% of the messages in each 
category. Our metric does not allow comparison between categories. 

Table 11 



Individual- Level Teamwork Process Scales (iV=81) 



Process 


Mean 


SD 


Minimum 


Maximum 


Adaptability 


.11 


.09 


.00 


.38 


Coordination 


.10 


.12 


.00 


.66 


Decision making 


.10 


.11 


.00 


.43 


Leadership 


.10 


.10 


.00 


.46 


Interpersonal 


.10 


.15 


.00 


1.07 


Communication 


.49 


.38 


.01 


2.15 
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Teamwork Process Reliabilities 

The five team process scales were subjected to reliability analysis using 
item raw scores (counts of times utilized), transformed item scores (proportion 
scores), and reduced scale proportion scores. There is an issue of 
independence of the item scores because a message could be selected by a 
participant more than once. In most examples of testing, an item can be 
answered only once. Thus, we explored a variety of reliability measures. We 
also looked at both parametric and nonparametric statistics, two of which we 
report here. 

Improvements for each scale in inter-item consistency were made by 
eliminating from one to three items from each message category, based on low 
item-total correlations. These exclusions included eliminating item #6 from 
the adaptability scale; items #16, #17, and #19 from the coordination scale; 
items #26 and #27 from the decision making scale; items #12 and #21 from the 
leadership scale; and items #48 and #49 from the interpersonal scale. These 
results can be seen in Table 12. Cronbach's alpha coefficients ranged from .51 
for adaptability to .76 for interpersonal in the reduced scales, 
indicating moderate inter-item consistency, with the interpersonal scale 
showing the greatest internal consistency. 

Additionally, nonparametric statistical tests indicate moderate but 
significant agreement between the items on each scale. The reduced version of 
each scale was analyzed using Kendall's coefficient of concordance (W 
statistic; see Hays, 1973). The coefficient of concordance indicates the extent to 
which judges (or items) agree in their ranking of subjects. It has a range from 
0 to 1, with zero indicating no agreement whatsoever, and one indicating total 
agreement. The statistic follows a chi-square distribution with (N-l) degrees of 
freedom, where N = number of subjects being judged. The null hypothesis that 
there is no agreement among the items can thus be tested, provided the 
number of items is roughly greater than or equal to 8 (Hays, 1973, p. 803), 
which is satisfied for all but one of these scales. The results of this analysis 
are presented in Table 13. The coefficients ranged from .24 to .31. In each 
case, the coefficient of concordance is significantly different from zero at p < 
.001. indicating agreement among the items. Thus, both parametric and non- 
parametric procedures show moderate consistency and reliability for each of 
the five teamwork process scales. 
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Table 12 

Summary of Reliability Analysis for Teamwork Process Scales 



Teamwork 
process scale 


Items 


A (11 H O f" TTl DTlfc 
X iLiJ liO LlJLACll LS 


Cronbach's 


Adaptability 


4,6,7,22,42,43,44,45,46,50 


Use raw values 


.49 




4,6,7,22,42,43,44,45,46,50 


Use proportion scores 


.43 




4,7,22,42,43,44,45,46,50 


rCemove #b 


.01 


Coordination 


9,11,13,14,15,16,17,18,19,24 


Use raw values 


.62 




9,11,13,14,15,16,17,18,19,24 


Use proportion scores 


.57 




9,11,13,14,15,18,24 


xtemove ffio, ffi/, ffiy 


.DO 


Decision making 


1,3,25,26,27,28,29,30,3 1,32 


T T 1 

Use raw values 


.44 




1,3,25,26,27,28,29,30,31,32 


Use proportion scores 


.61 




1,3,25,28,29,30,3 1,32 


Remove #26, #27 


.59 


Leadership 


2,5,8,10,12,20,21,23,33,34 


Use raw values 


.34 




2,5,8,10,12,20,21,23,33,34 


Use proportion scores 


.57 




2,5,8,10,20,23,33,34 


Remove #12, #21 


.63 


Interpersonal 


35,36,37,38,39,40,41,47,48,49 


Use raw values 


.57 




35,36,37,38,39,40,41,47,48,49 


Use proportion scores 


.75 




35,36,37,38,39,40,41,47 


Remove #48, #49 


.76 



Table 13 



Kendall's Coefficients of Concordance for Teamwork Scales 



Process 


W statistic 


Chi-square 


df 


P< 


Adaptability 


.24 


173.51 


80 


.001 


Coordination 


.31 


170.92 


80 


.001 


Decision making 


.29 


184.05 


80 


.001 


Leadership 


.24 


154.94 


80 


.001 


Interpersonal 


.30 


191.98 


80 


.001 
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Self-regulation Measures 

In addition to teamwork process scores, measures of self-regulation 
activity were computed for each subject. These included measures of effort, 
worry, cognitive strategies, self-checking, planning, and awareness. The 
values for each of these scales were calculated by summing the items from 
that scale. The means, standard deviations, number of items, and coefficient 
alpha reliabilities for each metacognitive scale are presented in Table 14. In 
general, reliabilities for the metacognitive measures are acceptable, with a low 
of .64 for the awareness scale to a high of .84 for the effort scale. 

Additional Measures 

Two questions presented to each subject dealt with the issue of fixed-pie 
bias, or a tendency to see the negotiation as a win-lose proposition. On the first 
question, regarding the management team's most important issue in the 
negotiation scenario, 76.5% of the subjects showed a fixed-pie bias. An even 
greater percentage (84.0%) indicated a fixed-pie bias on the second question, 
dealing with the management team's least important issue. These results are 
consistent with our prior data on novices (O'Neil et al„, 1994). 

We were also concerned with whether subjects understood what the 
priorities of their own negotiating team were. Of the 81 subjects in the study, 
71 (86.4%) indicated that they accurately comprehended their team's priorities. 
Moreover, since the negotiation within teams was intended to be among 
anonymous participants, we inquired regarding subjects' knowledge of the 

Table 14 



Descriptive Statistics and Reliabilities for Metacognitive Scales 



Metacognitive 
process 


Mean 


SD 


# of Items 


Cronbach's 
alpha 


Effort 


20.78 


3.46 


6 


.84 


Cognitive strategies 


17.15 


3.59 


6 


.70 


Worry 


9.81 


3.52 


5 


.74 


Awareness 


15.78 


2.82 


5 


.64 


Planning 


18.78 


3.66 


6 


.79 


Self-checking 


13.43 


3.33 


5 


.74 



.8 
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identity of the other members in their team. In general, subjects were mostly 
unaware of the identity of their other team members (Table 15). The overall 
mean was 1.79 indicating average knowledge somewhere between 1 "Not at all" 
and 2 "Somewhat." 

Finally, information regarding subjects' perceptions of the simulation 
program, message effectiveness, and computer literacy was solicited. 
Generally, subjects found the program to be easy to use (M = 3.40, SD = .83; 
[82.7% of subjects indicated either "often" or "almost always" as opposed to 
stating "almost never" or "sometimes"]). Though the messages didn't always 
capture what the subjects wanted to say CM = 2.63, SD = .90 [46.9%]), they were 
considered to be effective (M = 3.27, SD = .82 [81.5%]) and coherent (M = 3.52, SD 
= .73 [86.4%]). Additionally, the subjects indicated high levels of computer use 
CM = 3.37, SD = .87 [81.5%]). 

Team-Level Measures 

Each of the individual-level teamwork process measures was used to 
calculate comparable team-level scores. The individual scores for adaptability, 
coordination, decision making, leadership, interpersonal, and error messages 
were summed among the three members of each team to generate a team 
score for each process. A total team communication score was created by 
subtracting the total team error messages from the sum of the five team-level 



Table 15 

Means and Standard Deviations for Teammate 
Identity 



Variable 


Mean 


SD 


Knowledge of leader 


1.83 


1.22 


Knowledge of member 1 


.70 


1.14 


Knowledge of member 2 


1.83 


1.19 


Overall 


1.79 


1.02 



Note. These values were computed excluding self- 
knowledge (i.e., Leaders were excluded from the 
Knowledge of Leader question, Mis were excluded 
from the Knowledge of Member 1 question, and M2s 
were excluded from the Knowledge of Member 2 
question). 

4 ( J 
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teamwork process scores. Means, standard deviations, and intercorrelations 
for each of the team-level teamwork process measures are presented in 
Table 16. Communication is a composite (simple sum) of the other five team 
process measures, which are restricted to a 0-1 range. However, the range for 
Communication is from .22 to 3.05. Unlike the other team process measures, 
Communication is a composite, rather than a proportion. 

Outcome Measures 

The negotiation scenario generated a total of four team-level outcome 
measures: team performance score, whether an agreement was reached, type 
of agreement reached, and time spent on the negotiation. Team performance 
scores were calculated for teams reaching agreement as the total point value of 
the agreed-upon offer. For teams not reaching an agreement, a team 
performance score was computed by taking the total point value of the inverse 
of the last offer submitted by the negotiating team. Upon inspection, we found 
one team that completely misunderstood the concept of the scenario, which 
resulted in a team score of zero. Through discussions with the members of 
this team, we realized they mistakenly believed that a score of 0 was their ideal 
result, as opposed to the instructions indicating the objective was to maximize 
the points for their team according to their stated priorities. As a result, this 
team was excluded, leaving data for 26 teams for all team analysis. 



Table 16 

Descriptives and Intercorrelations for Team-Level Teamwork Process Measures 





Process 


Mean 


SD 


2 


3 


4 


5 


6 


1. 


Adaptability 


.33 


.20 


.69*** 


g2*** 


.49** 




*** 


2. 


Coordination 


.28 


.21 




72*** 


.65*** 


.45* 


.88*** 


3. 


Decision making 


.29 


.20 






.66*** 


.41* 


gg*** 


4. 


Leadership 


.28 


.18 








.37 


77*** 


5. 


Interpersonal 


.24 


.15 










.66*** 


6. 


Communication 


1.38 


.76 













*p<.05. **p<.01. ***/?<. 001. 
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The outcome measure of agreement was coded dichotomously (1 = 
agreement; 0 = no agreement). The great majority of teams reached an 
agreement within the 8-round negotiating session (85%). Agreement type was 
determined by the value of the overall score measure. Teams with overall 
scores greater than 120 received a score of 1, indicating an integrative 
agreement. Teams with overall scores less than or equal to 120 received a 
score of 0, indicating a distributive agreement. Time spent in the negotiation 
scenario was captured by the program for each team and is reported in 
minutes. Descriptive statistics for each of these measures can be found in 
Table 17. 

Tests of Hypotheses 

We hypothesized several relationships between the teamwork processes 
and outcome measures, as well as some relationships among the outcome 
measures themselves. Our unit of analysis was the team (JV=26), not the 
individual student. Our first hypothesis was that team processes would be 
positively associated with team performance. This hypothesis was generally 
supported by the direction and moderate magnitude of the correlations between 
these measures; however, few are statistically significant. Failure to find 
statistical significance for these correlations is not surprising considering the 
relatively low power of this test given a sample size of only 26. According to 
Cohen (1992), for a correlation coefficient test of significance with a power of .80 
to detect even a small effect size (defined by Cohen to be > .10) at the .05 level of 
significance would require a sample size of N = 783. To detect a medium effect 
size (e.g., > .30), a sample size of N = 85 would be required. In the current 
study, only large effect sizes (e.g., > .50) could be expected to be detected 
(necessary sample size, N = 28). 

Table 17 



Descriptives Statistics for Team-Level Outcome Measures (N = 26) 



Outcome measure 


Mean 


SD 


Minimum 


Maximum 


Performance score 


121.73 


21.07 


50 


150 


Agreement 


.85 


.37 


0 


1 


Agreement type 


.50 


.51 


0 


1 


Time in negotiation (min) 


28.31 


9.02 


13.00 


44.00 
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Table 18 provides the correlations between the teamwork processes and 
outcome measures (Hypothesis 1). Correlation coefficients for team 
performance and adaptability, coordination, and leadership range from .29 to 
.31. The correlation between team performance and team communication is 
higher (r = .36) and approaches significance (p = .07). Only team decision 
making was significantly related to team performance ( r = .41; p < .04). The 
interpersonal teamwork process showed no relationship to team performance 
(r = .03; p > .88); however, it did show a strong negative relationship with 
reaching an agreement (r = - .52; p < .01). Apparently, though most teams 
reached an agreement, those teams that failed to reach an agreement (4 of 26 
teams, or 15%) utilized more interpersonal messages than did teams that 
reached an agreement. This statement is implied by the negative correlation 
between agreement and the interpersonal team process measure (-.52**). 
However, the mean values of the interpersonal measure for each of these 
groups (agreement vs. non-agreement) are listed in Table 19: These differences 
are significant at the p < .01 level (t value = 3.00, df= 24). 

Similar results are found for our additional hypothesis that teamwork 
processes would be positively associated with quality of agreement. Again, 
moderate positive but nonsignificant correlations were found between type of 
agreement and adaptability, coordination, leadership, and communication (r's 



Table 18 

Correlations Between Team Processes and Outcome Measures (N = 26) 





Performance 




Agreement 


Time in 


Team process 


score 


Agreement 


type 


negotiations 


Adaptability 


.31 


-.24 


.24 


72* ** 


Coordination 


.29 


-.08 


.15 


.65*** 


Decision making 


.41* 


-.04 


.41* 


.69*** 


Leadership 


.32 


-.02 


.19 


.65*** 


Interpersonal 


.03 


-.52** 


-.05 


.56*** 


Communication 


.35 


-.20 


.24 


.80*** 



*p<.05. **p<.01. ***p<.001 
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Table 19 

Means and Standard Deviations Between Teams With and 
Without Agreement 



Team outcome 


Mean 


SD 


SE 


Agreement (iV=22) 


.20 


.13 


.03 


Non-agreement GY=4) 


.42 


.16 


.08 



= .24, .15, .19, and .24 respectively). Decision making showed a significant 
positive relationship with agreement type (r = .41; p < .04), but as with team 
performance, the interpersonal teamwork process showed no relationship to 
quality of agreement (r = -.04; p > .81). 

There was more support for our prediction of a positive relationship with 
the teamwork processes and time involved in negotiation. All of the 
correlation coefficients between time involved in negotiation and the teamwork 
processes are positive and significant beyond the p < .001 level. They range 
from a low of .56 for the interpersonal scale to a high of .80 for the 
communication measure. Thus, not surprisingly, more communication of ad 
types among the team members is related to longer time involved in the 
negotiation process. 

Our second hypothesis predicted that team performance would be 
negatively related to time in negotiations. Results for this prediction are 
mixed. Although time in the negotiation scenario was negatively related to 
reaching an agreement (r = -.40; p < .05), the correlation between time and 
team performance was positive but not significant (r = .28, p > .16). Similarly, 
results indicate no relationship between time and agreement type (r = .10; p > 
.64). Thus, teams that reached agreement took less time in the negotiations, 
as we predicted. However, less time did not relate to significantly higher team 
performance values or better agreements. 

The third hypothesis proposed a negative relationship between team 
performance and the degree of fixed-pie bias exhibited by the team. Team-level 
fixed-pie bias was calculated as the sum of bias exhibited by the three members 
of the team on two questions; thus, the possible range for this team-level 
variable is from 0 (no bias by any member on either of the two questions) to 6 
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(all three members exhibit bias on both of the fixed-pie questions). However, 
due to the large amount of bias shown at the individual level (recall 76.5% on 
the first question and 84% on the second question), the team bias score ranged 
from 3 to 6 (M = 4.81; SD = 1.02). Correlations between team performance 
measures and team level bias did not support our hypotheses. Contrary to our 
prediction, the relationship between team performance and bias (r = .29; p > 
.15) was positive but not significant. Likewise, no significant relationship was 
found between bias and type of agreement (r = .04; p > .85). 

We also proposed relationships between the self-regulation measures 
(effort, cognitive strategies, worry, awareness, planning, and self-checking) 
and team performance measures (Hypothesis 4). All but worry were predicted 
to show positive correlations with the team performance measures (Table 20). 
Though not all of the proposed relationships were exhibited, we found support 
for the measures of effort and cognitive strategies. Teams exhibiting more 
effort on the task showed higher team performance (r = .57; p < .005) and better 
agreements (r = .57; p < .005). Likewise, teams higher on the cognitive strategy 
measure showed higher performance (r = .38; p = .05) and better agreements (r 
= .36; p <.07), though this relationship only approached statistical significance. 
As seen in Table 20, none of the other relationships are significant, though 
many are in the proposed directions and are of low to moderate magnitudes, 
magnitudes that are consistent with earlier research in this area (e.g., 
Brannick et al., 1993). As noted earlier, one of the limitations of sample sizes 



Table 20 

Correlations Between Metacognitive and Outcome Measures (N = 26) 





Performance 




Agreement 


Time in 


Metacognitive scale 


score 


Agreement 


type 


negotiations 


Effort 


.57** 


.48* 


.57** 


.08 


Cognitive strategies 


.38* 


.23 


.36 


.15 


Worry 


-.25 


-.21 


-.33 


.25 


Awareness 


.22 


.23 


.16 


-.03 


Planning 


.30 


.26 


.28 


.26 


Self-checking 


-.15 


.08 


-.17 


.23 


*p<.05. **p<.01. 


< .001 
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as small as the one used in these group level analyses (N = 2t3) is that all but 
very large effects are not likely to be detected and recognized as statistically 
significant. Clearly, additional investigations with more groups are desirable 
to ferret out small to moderate effects of teamwork processes and 
metacognitive activities on team performance outcomes. 

Discussion « 

The results of this study indicate that the measurement of teamwork 
processes can be accomplished in a reasonably reliable and much more time- 
efficient manner than that of earlier approaches. The entire negotiation 
procedure, including individual and team process and outcome scoring and 
interpretation, took place in less than one extended class period. This kind of 
improvement in the timeliness of data collection and analysis without a loss in 
the reliability and validity of such measures greatly enhances the ability of 
researchers to address pressing issues and speeds up the accumulation of 
knowledge in the area. 

In addition, as an initial small-scale investigation, this study provides 
promise that larger scale experimentation may provide more stable and 
statistically significant estimates of the relationships between teamwork 
measures, such as adaptability coordination, decision making, leadership, 
and interpersonal processes, and outcome measures, such as whether 
negotiated agreements are reached, the type and quality of those agreements, 
and the time involved in reaching such agreements. 

Moreover, we have shown that the use of technology, specifically 
computerized networked simulations, can be effectively incorporated in 
assessing the relationships among these relevant issues. Students found the 
simulation to be interesting, easy to use, effective, and enjoyable. Few other 
testing procedures enjoy such enthusiastic, positive appraisal from the test- 
taking populace. Acceptance of and support for the procedure by students can 
facilitate additional investigations in the area and may lead to additional 
improvements in the way teamwork processes and other relevant constructs 
are measured. 

Improving the ability to assecs teamwork skills and their relationships to 
performance outcomes will serve not only the interests of educational 
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researchers but also the interests of training and development specialists, and 
American industry as well. With the increasing importance placed on 
teamwork skills in the private sector, and the desire to encourage and develop 
such abilities in. young adults entering the workforce, the development of 
efficient teamwork taxonomies and assessment procedures is of critical 
importance. This study provides direction in pursuing this objective and 
moves us closer to accomplishing that goal. 

In conclusion, existing measures of team processes are labor intensive 
(think-aloud protocols or ratings of videotaped team sessions) and thus are not 
timely. Our approach suggests the feasibility of computer-scoring and 
reporting of team processes in real time. We plan to conduct more statistical 
analysis to understand more clearly the relationship between team processes 
and outcomes. Each student, when finished with the simulation, would have 
a score for each team process (e.g., coordination or leadership) as well as a 
score of the team outcome. Our software approach has been designed to be 
domain independent and thus should transfer to other computer-based team 
environments. Such a reliable and valid measure would offer the capcMlity of 
assessing quickly team processes and outcomes in collaborative learning 
environments (K-12) or team training in industry or the military. 
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Appendix 1 



Algorithm of Simulated Management Team's Response 



Computer-Simulated Offers Algorithms 

Notation/defi nitions : 

H[l] = human offer on issue 1 
H[2] = human offer on issue 2 
H[3] = human offer on issue 3 

Inverse: reflection about issue E. The inverse of A = I; of C = G; of E = E. 
Mirror: one-for-one match. The mirror of A = A; of C = C; of E = E. 

Computer Offer on Issue 1: 

If H[l] < E, computer offer = inverse of H[3] 
If H[l] >= E, computer offer = mirror of H[3] 

For example: 



For H[l] < E: 

Hf31 Issue 1 Offer 



A 


I 


B 


H 


C 


G 


D 


F 


E 


E 


F 


D 


G 


C 


H 


B 


I 


A 


-H[l] >= 


E: 


W31 Is 




A 


A 


B 


B 


C 


C 


D 


D 


E 


E 


F 


F 


G 


G 


H 


H 


I 


I 
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Computer Offer on Issue 2: 

If H[2] < E, computer offer = inverse of H[2] 
If H[2] >= E, computer offer = mirror of H[2] 



For example: 

Hf21 Issue 2 Offer 



A 


I 


B 


H 


C 


G 


D 


F 


E 


E 


F 


F 


G 


G 


H 


H 


I 


I 
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Computer Offer on Issue 3: 

If H[l] < E, computer offer = inverse of H[l] 
If H[l] >= E, computer offer = mirror of H[l] 

For example: 

For H[l] < E: 

HJH Issue 3 Offer 



A 


I 


B 


H 


C 


G 


D 


F 


E 


E 


F 


F 


G 


G 


H 


H 


I 


I 



For H[l] >= E: 

Hill Issue 3 Offer 



A 


I 


B 


H 


C 


G 


D 


F 


E 


E 


F 


F 


G 


G 


H 


H 


I 


I 
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HyperCard code to implement the above. 



computeScore 



function computeScore usOrThem, issuelLine, issue2Line, issue3Line 
if usOrThem = 1 then 

-- compute score for us (human) 

-- line 1 = highest score, line 9 = lowest (0) 

— -15 * lineNo + 135 = issue 1 score 

-- -10 * lineNo + 90 = issue 2 score 

-- -5 * lineNo + 45 = issue 3 score 

put (-15 * issuelLine + 135) + 

(-10 * issue2Line + 90) + 

(-5 * issue3Line + 45) into score 

else 

-- compute score for them (computer) 

-- line 9 = highest score, line 1 = lowest (0) 

5 * lineNo - 5 = issue 1 score 
-- 10 * lineNo - 10 = issue 2 score 
-- 15 * lineNo - 15 = issue 3 score 
put (5 * issuelLine - 5) + 

(10 * issue2Line - 10) + 

(15 * issue3Line - 15) into score 

end if 

return score 
end computeScore 
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computeResponseToOurOf f er 



on computeResponseToOurOf fer 

global glssuelOf ferUsLine -- line no in field 

global glssue20f ferUsLine -- line no in field 

global glssue30f ferUsLine -- line no in field 

global glssuelOf ferThemLine -- line no in field 
global glssue20f ferThemLine -- line no in field 
global glssue30f ferThemLine -- line no in field 



-- 'our' means the offer we (ie, computer users) are giving to 
hypothetical 

-- them (computer simulation) 

-- compute offer 1 response 

if glssuelOf ferUsLi;.^ < 5 then 

-- simulated rcy.^.r.-ie = inverse on issue 3 of human 
put 10 - glssue30f ferUsLine into glssuelOf ferThemLine 

else 

-- simulated response = mirror on issue 3 of human 
put glssue30f ferUsLine into glssuelOf ferThemLine 
end if 

-- offer 2 response 

if glssue20f ferUsLine < 5 then 

-- simulated response = inverse on issue 2 of human 
put 10 - glssue20f ferUsLine into glssue20f ferThemLine 

else 

-- simulated response = mirror on issue 2 of human 
put glssue20f ferUsLine into glssue20f ferThemLine 
end if 

-- offer 3 response 

if glssuelOf ferUsLine < 5 then 

-- simulated response = inverse on issue 1 of human 
put 10 - glssuel ferUsLine into glssue30f ferThemLine 

else 

-- simulated response = mirror on issue 1 of human 
put glssuelOf ferUsLine into glssue30f ferThemLine 
end if 



end computeResponseToOurOf fer 
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Appendix 2 

Messages Grouped by Category and by Message Number 

Items requiring input from participants are denoted by X, Y, Z, and 
brackets. For example, in message number 4, "We should offer X on 
[W/H&W7P]," X indicates that the participant needs to enter a value, and 
[W/H&W/P] indicates that the participant needs to choose Wages, Health and 
Welfare, or Pension. Thus, an example of the fully resolved message would be 
"We should offer 120 on Wages." Note that under the Agreement Column, "P" 
denotes primary agreement, "S" denotes secondary agreement, and "R" 
denotes reconciled by a third rater. 
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Messages Grouped by Category 






Msg. # 


Message 


Agreement 


Adaptability 



4 We should offer X on [W/H&W/P]. 

6 If we give on [W/H&W/-M, they might give on [W/H&W/P]. 

7 How [low/high] should we go on [W/H&W/P]? 
22 What issue is [leastfmost] important to them? 

42 I agree with [M1/M2/LD]. 

43 I disagree with [M1/M2/LD]. 

44 What do you think, [M1/M2/LD]? 

45 What does the team think? 

46 Any ideas? 

50 Should we accept their offer? 



P 
P 
P 
P 
P 
P 
P 
R 
P 
P 



Coordination 

9 We should each figure out our preferences, and then share them. 
11 We need to hurry to complete this round. 

13 We only have X rounds left. 

14 Keep track of the time. 

15 We need to get an agreement. 

16 I think we should be making offers we seriously think they might take by 
the 6th round. 

17 Should we hold tough until near the end, or start moving toward a 
compromise now? 

18 We should compromise now. 

19 We should coordinate our activities. 

24 We should sequence our offers so as to gain agreement. 



R 
P 
P 
S 
S 

s 



s 
p 
s 



Decision making 

1 What if we offer X-Y-Z? P 
3 What if we offer X on [W/H&W/P]? R 

25 We should go higher on [W/H&W/P] because it's OUR [lstf2nd/3rd] P 
priority. 

26 We should go higher on [W/H&W/P] because it's THEIR [lstf2nd/3rd] P 
priority. 

27 If we don't go higher on [W/H&W/P1, then we'll be sacrificing our P 
position. 
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Msg. # 



Message 



Agreement 



Decision making (continued) 

28 If we don't go higher on [W/H&W/P], then we'll get a bad deal. 

29 We should go lower on [W/H&W/P] because it's OUR [lst/2nd/3rd] 
priority. 

30 We should go lower on [W/H&W/P] because it's THEIR [lst/2nd/3rd] 
priority. 

31 We should go lower on [W/H&W/P] because we can afford to. 

32 We should go lower on [W/H&W/P] because it increases our chances of 
an agreement. 



P 
P 



P 
P 



Interpersonal 

35 We should all agree on the final offer. P 

36 We have to work together on this. P 

37 Our members are counting on us to be an effective team. P 

38 With everyone's contribution, I'm sure we're going to be successful. P 

39 I think everyone's helping. Let's keep it up. P 

40 Help me out. P 

41 Do you need any help? P 

47 Good idea. P 

48 I think you've got the right idea. P 

49 It's important that we all contribute. P 



Leadership 

2 Let's offer X-Y-Z. 

5 We should [give/hold/raise] on [W/H&W/P]. 

8 We should maximize our total points. 

10 Let's go [higher/lower] and see what they do. 

12 Let's make a decision now. 

20 Our [lst/2nd/3rd] priority is [W/H&W/P]. 

21 Let's assume the management team's [lst/2nd/3rd] priority is 
I W/H&W/P]. 

23 Let's think about what our strategy should be. 

33 Send the offer. 

34 Should we send our offer? 



P 
P 
P 
P 
P 
P 
P 

P 
P 
P 



Other 

51 I sent the wrong message (clicked on wrong button). 
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Messages Sorted by Message Number 



Msg. # Message 



Agreement 



1 Decision 
making 

2 Leadership 

3 Decision 
making 

4 Adaptability 

5 Leadership 

6 Adaptability 

7 Adaptability 

8 Leadership 

9 Coordination 

10 Leadership 

11 Coordination 

12 Leadership 

13 Coordination 

14 Coordination 

15 Coordination 

16 Coordination 

17 Coordination 

IS Coordination 

19 Coordination 

20 Leadership 

21 Leadership 

22 Adaptability 

23 Leadership 

24 Coordination 

25 Decision 
making 



What if we offer X-Y-Z? 
Let's offer X-Y-Z. 

What if we offer X on [W/H&W/P]? 

We should offer X on (W/H&W/P] . 

We should [give/hold/raise] on [W/H&W/P] . 

If we give on [W/H&W/P], they might give on [W/H&W/P]. 

How [low/high] should we go on [W/H&W/P]? 

We should maximize our total points. 

We should each figure out our preferences, and then share them. 

Let's go [higher/lower] and see what they do. 

We need to hurry to complete this round. 

Let's make a decision now. 

We only have X rounds left. 

Keep track of the time. 

We need to get an agreement. 

I think we should be making offers we seriously think they might take 
by the 6th round. 

Should we hold tough until near the end, or start moving toward a 
compromise now? 

We should compromise now. 

We should coordinate our activities. 

Our [lst/2nd/3rd] priority is [W/H&W/P]. 

Let's assume the management team's [lst/2nd/3rd] priority is 
[W/H&W/P] . 

What issue is [least/most] important to them? 

Let's think about what our strategy should be. 

We should sequence our offers so as to gain agreement. 

We should go higher on [W/H&W/P] because it's OUR [lst/2nd/3rd] 
priority. 
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Msg. # 


Message 


Agreement 


26 


Decision 
making 


We should go higher on [W/H&W/P] because it's THEIR [lst/2nd/3rd] 
priority. 


27 


Decision 
making 


If we don't go higher on [W/H&W/P], then we'll be sacrificing our 
position. 


28 


Decision 
making 


If we don't go higher on [W/H&W/P], then we'll get a bad deal. 


29 


Decision 
making 


We should go lower on [W/H&W/P] because it's OUR [lst/2nd/3rd] 
priority. 


OA 


Decision 
making 


we snouiQ go lower on iw/nocw/r] uecause its ini^ixv ixsuznu/orcij 
priority. 


31 


Decision 
making 


We should go lower on [W/H&W/P] because we can afford to. 


32 


Decision 
making 


We should go lower on [W/H&W/P] because it increases our chances of 
an agreement. 


OO 


Leadership 


ucuQ tne oner. 




T AA/1 Awn nif\ 

jjeaaersmp 


OiiGuiu. Wc oenu our oiiei : 


O0 


Interpersonal 


we snouiQ ail agree on tne nnai oner. 


0£2 
OD 


Interpersonal 


we nave to worK togetner on tms. 


61 


Interpersonal 


~- vvt aiV^ A»»n n A Anil T" 1 r\ rr ATI lie T" f~\ f~\ /~\ a Tl atTA^TI ipa i a Q TY1 

uiir memuers are counting on uo to ue mi cTicvtivc team. 


OO 


Interpersonal 


mronrnno'c pnnfriKnfinn T*l*n c n t*o wj oVo crninCT trv nP on ppac cfnl 

vv 11*11 everyone o cuiitrmu.tiuii ? i m aurc wc ic guuig ue ouitcsoim. 


OQ 

ocf 


intcrpt?roonai 


T fVi i n lr ovorvnno'c V^olnincr T .pt"*Q IrooTl it" nn 

1 till 11 IV CVCljUllC O lleipiilg. JUCt O XVCCp 11/ ixp. 




Interpersonal 


neip me out. 


/II 
11 


niter personal 




4.9 


A A a r\+"a 1*n 1 1 +■ v 
n.UUp XfCLUHLXfj 


T acrrpp with FM1/M2/LD1 




a p C a U 1 1 1 C jr 


T Hisnrrrpp with rMl/M2/I,Dl 


44 


Adaptability 


What do you think, [M1/M2/LD]? 


45 


Adaptability 


What does the team think? 


46 


Adaptability 


Any ideas? 


47 


Interpersonal 


Good idea. 


48 


Interpersonal 


I think you've got the right idea. 


49 


Interpersonal 


It's important that we all contribute. 


50 


Adaptability 


Should we accept their offer? 


51 


Other 


I sent the wrong message (clicked on wrong button). 



o 
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Appendix 3 
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Appendix 4 

Self-regulation Questions 

1. I concentrated as hard as I could on the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

2. I wasn't happy with my performance. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

3. I thought through the meaning of the negotiation issues before I began sending messages. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

4. I noted the negotiation issues I wasn't sure of so I could go back if there was time. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

5. I did not give up, even if the negotiation task was hard. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

6. I was aware of which thinking technique or strategy to use and when to use it. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

7 . I tried to do my best on the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderataly So 

(D) Very Much So 

8 . I put forth my best effort. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 
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9 . I made sure I understood just what had to be done and how to do it. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

10. I developed a plan for the solution of the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

11. I attempted to discover the main ideas in the negotiation issues. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

12. I worked hard on the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

13. I did not feel very confident about my performance on the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

14. I attempted to use the big or major principles to complete the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

15. I was concerned about what would happen if I did poorly. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

16. I asked myself how the negotiation issues related to what I already knew. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

17. I corrected my errors. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 
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18. I was aware of my ongoing thinking processes. 
(.A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

19. I determined how to solve the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

20. I kept working, even on difficult negotiation issues. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

21. I was aware of my own thinking. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

22. I was afraid I should have studied the tutorial more for this negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

23. I went over what I knew about the negotiating issues (Wages, Health & Welfare, 
Pension). 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

24. I felt regretful about my performance on the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

25. I thought my performance was bad, so everyone, including myself, would be 
disappointed. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 



r, 
( O 
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26. I tried to understand the negotiation issues before I attempted to send messages. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

27. On difficult negotiation issues, I spent more time trying to understand them. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

28. I tried to determine what the negotiation task required. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

29. I checked my performance while I was doing the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

30. I selected and organized relevant information to perform the negotiation task. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

31. I was aware of my trying to understand the negotiation issues before I attempted to send 
messages. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

32. I judged the correctness of my performance. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

33. I was aware of the need to plan my course of action. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

34. I tried to understand the goals of the negotiation task before I attempted to send messages. 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

( U 
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Fixed-Bias Questions 

1. What issue do you think was the MOST IMPORTANT to the MANAGEMENT team in 
the negotiation? 

*(A) Pension 

(B) Health & Welfare 

(C) Wages 

2. What issue do you think was the LEAST IMPORTANT to the MANAGEMENT team in 
the negotiation? 

(A) Pension 

(B) Health & Welfare 
*(C) Wages 

Manipulation Check Questions 

3. YOUR team's priorities were: 

(A) 1st: Health & Welfare, 2nd: Pension, 3rd: Wages 

(B) 1st: Pension, 2nd: Health & Welfare, 3rd: Wages 

(C) 1st: Pension, 2nd: Health & Welfare, 3rd: Wages 
*(D) 1st: Wages, 2nd: Health & Welfare, 3rd: Pension 

4. Do you know who Ml is? 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

5 . Do you know who M2 is? 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

6 . Do you know who LD is? 

(A) Not at All 

(B) Somewhat 

(C) Moderately So 

(D) Very Much So 

7 . Did the messages capture what you wanted to say? 

(A) Almost Never 

(B) Sometimes 

(C) Often 

(D) Almost Always 



Correct answer. 
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8. Did you put much effort into finding the message that most closely matched what you 
wanted to say? 

(A) Almost Never 

(B) Sometimes 

(C) Often 

(D) Almost Always 

9. Were you able to follow the "electronic conversation"? 

(A) Almost Never 

(B) Sometimes 

(C) Often 

(D) Almost Always 

10. Did you find it easy to use the computer system? 

(A) Almost Never 

(B) Sometimes 

(C) Often 

(D) Almost Always 

11. Do you use computers? 

(A) Almost Never 

(B) Sometimes 

(C) Often 

(D) Almost Always 

12. Was there anything you really wanted to say but didn't find in the messages? If so, 
please type in the message(s) in the space below. Enter as many messages as you like, 
starting with the most important ones. 
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Appendix 5 

Computer Screen Shots of Individual and Team Performances 



Teamwork Skills 



This simulation was designed to measu^ some of the processes that have been 
identified as being important for team negotiation skills. In particular, five 
processes were measured: (1) adaptability - recognizing problems and responding 
appropriately, (2) coordination - organizing team activities to complete a task on 
time, (3) decision-making - using available information to make decisions, (4) 
interpersonal - interacting cooperatively with other team members, (5) leadership 
- providing direction for the team, and (6) communication - the overall exchange of 
clear and accurate information. 

The next screen provides a description of each process and shows how many messages 
you sent. We used the number and type of messages as the means to measure team 
processes. Past research has shown that members of successful teams generally use 
team processes more often than unsuccessful teams. Note that 'Communication' 
represents the total number of messages sent, and roughly represents how active as 
a team member you were. 
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INDIVIDUAL Performance 



Adaptability 

Recognizing problems and responding appropriately. 
Coordination 

Organizing team activities to complete a task on time. 
Decision-Making 

Using available information to make decisions. 
Interpersonal 

Interacting cooperatively with other team members. 
Leadership 

Providing direction for the team. 
Communication 

The overall exchange of clear and accurate 
information. 



The number of message(s) 
you sent: 1 2 



The number of message(s) 
you sent: 3 



The number of message(s) 
you sent: 6 



The number of message(s) 
you sent: 1 3 



The number of message(s) 
you sent: 1 1 



The number of message(s) 
you sent: 44 



OK 



SO 



er|c 
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TEAM Performance 



Whereas the previous screen showed you the number of messages sent by you, 
the next screen will show the number of messages sent by your TEAM. That 
is, for each team process (e.g., adaptability), you will see the total number of 
messages sent by you and your teammates. 

Past research has shown that members of successful teams generally use team 
processes more than unsuccessful teams. 
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TEAM Performance 



Adaptability 

Recognizing problems and responding appropriately. 

Coordination 

Organizing team activities to complete a task on time. 
Decision-Making 

Using available information to make decisions. 
Interpersonal 

Interacting cooperatively with other team members. 
Leadership 

Providing direction for the team. 
Communication 

The overall exchange of clear and accurate 
information. 



The number of message(s) 
your TEAM sent: 40 



The number of message(s) 
your TEAM sent: 22 



The number of message(s) 
your TEAM sent: 28 



The number of message(s) 
your TEAM sent: 17 



The number of message(s) 
your TEAM sent: 50 



The number of message(s) 
your TEAM sent: 155 







? 




OK 
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TEAM Negotiation Perforr-since 



The next screen will show you how well your team did on your negotiation task. 
You will be shown three things: 

(1 ) Whether or not your team reached an agreement with the management team. 

(2) If your team reached an agreement - the type of agreement. This will be 
explained in more detail on the next screen. 

(3) How long your team took to complete the negotiations. 



OK 
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CRESST Final Deliverable 



TEAM Negotiation Performance 



Agreement Reached? 

Whether or not your team reached an agreement 
with the management team. 

Type of Agreement 

We built into the negotiation task two kinds of 
agreement, integrative and distributive. 

Integrative : "win-win" agreement. Your team 
gets what you want, the management team gets 
what they want. 

Distributive : "win-lose" agreement. One team 

gets what they want at the expense of the other team. 

Elapsed Time 

The time it took for your team to complete the 
negotiation task. 



Agreement 
reached. 



Integrative 



28 min., 51 sec. 



OK 



3 



Q 1 
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Appendix 6 



List of messages and comments participants entered. Entries are listed by 
participant ID number. 

#3 

This was a fun program. I hope you are successful of selling this program 
#4 

What do you want me to do about the other team. 
#6 

We should go lower so that they will agree ,where if we were 
#7 

I think we did real well. .there were times of disagreement but we fought back. 
I liked it 

#8 

Send the message now! 
You're wrong 

#14 

I think ml/m2/ld should put more effort in? 
#18 

I am sorry. 

Lets send this offer 

what was the offer again? 

why? 

#19 

I really dislike your idea. Don't you know that if you go lower, the team will get 
less than it deserves. If we go higher, they won't lower the prices. 

#20 

why don't we do this,...? 
why are you so stubborn? 
stop messing around! 
why such a high offer? 
send this offer or else...! 
come on, pleeeeze offer this!? 
fine, do it without me! 
some offer!! 
this is all your fault! 
forget it! 

darn!! or 

GO 



80 



CRESST Final Deliverable 



#34 

Why didn't you people help me out. 

You all mess around to much. THANKS A LOT!!!!!!! 

#35 

the leader s**ks 
you talk too much 
send it you freak! 
this is fun 
who are you 
you mama 

#36 

shut up 

then decide on something you DO like!!!!!!!! 
why did you do that 
go home!!!!! 

#43 

shut up 
that's dumb 
your a moron 
geat P*king idea 



#53 

IT WOULD PROBABLY BE A BETTER SYSTEM IF YOU COULD ACTUALLY 
TYPE IN YOUR OWN RESPONSES TO OTHER TEAMMATES. I WAS 
UNDER THE IMPRESSION THAT THIS IS WHAT WE'D BE ABLE TO DO. 

#54 

Are you sure? 
#56 

don't send offer yet!!! 

send offer we both agree on the decision. 

#61 
yes 
no 

#62 

Well I felt that if you were able to write what you wanted to say in addition with 
the quick responses, It would make it a better program to use... 

#65 

I just wanted some more quick responses and maybe some messages with 
more user input. 
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#67 

is everything fine 

do you think changes should be made 
what can I do to make it better 

#68 

we need to start high and compromise from there 
#073 

I'm not too sure about that. 

On a 1-5 scale, what do you rate the offer they gave us? 
Why? 

Can we make a compromise? 

No 

Yes 

Do happen to know their priorities? 
Whose offer should I send, MI or M2? 

Are you sure they will accept that offer, looking at their last offer to us? 
#074 

The message-sending was fun, but I think that the negotiation situation 
should be clarified. At first, I was somewhat confused about what we were 
doing- luckily after the negotiations began I caught on. Maybe there should be 
a scenario so that students will better understand this activity. 

#75 

All I needed to say were in the programmed messages. 
#76 

Hello? Is anyone there 
#078 

I would like to suggest more versatility with messages being sent. I realize 
that giving us the opportunity to send whatever we wanted to say might result 
in some unimportant messages. But considering how responsible I think I 
am and the seriousness of this issue if it was in real life, I think that there 
should be a way that we can express ourselves if we cannot find an appropriate 
message. 

Why do you think that? 
#081 

What do you mean by...? 

I think they will agree with our offer. 

We should go higher on (wages/h&w/P) because they seem like they will go 
along with it. 

If we go higher on (W/H&W/P), then they might not agree and lower the entire 
thing. 
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CRESST Final Deliverable 



#83 
yes 
no 

Does everyone agree? 

[M1/M2/LD] needs to contribute more 

Sorry. 

#84 

You idiot! What did you do?!?!?!?!? 

We must hold tough. 

We should accept their offer. 

#085 

I really can't remember what I wanted to say but I felt that the process of 
communicating would proceed a lot faster and easier if the members could 
write their own comments since there were many times were I felt restricted 
by the set choices given to me. 

#086 

Yes and No answers 

Can you hurry before time runs out? 

What are our intentions? 

Let's deduct their reasoning. 

Please assess my responses. 

I need feedback from all of you. 

What are OUR disagreements, maybe we can learn from our mistakes. 
I would like to accept this offer? Agree? 
Remember to keep in mind our priorities. 

#87 . 

You are being contradictory 

Our offer is nearly the same as theirs. 

#088 

Next time it might go faster if the members of the team were able to discuss a 
real conversation through the computer instead of merely sending messages. 
This could lead people off-track however. 

#091 

I thought that probably the hardest part of the negotiations-besides the 
negotiations themselves--was actually finding the messages and their 
corresponding numbers. Even though it was printed on a sheet and popped up 
when the cursor touched it, it still took some time to find the one I actually 
wanted. I thought that it would have been better if there was one option where 
you could type in what you wanted to say, because there were things I wanted 
to say that were not listed as options. I felt limited in that way. Other than 
that, I felt that it was a good activity. 
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88 



Program Two, Project 2.1 



83 



#93 

be reasonable 
#094 

I wanted to tell some people to get with it and try to contribute and that they 
should offer something 

#96 
yes 
no 

I'm confused 
what should we do? 
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